In [1]:
!mkdir -p ~/agave/funwave-tvd-docker

%cd ~/agave

!pip3 install setvar

import re
import os
import sys
from setvar import *
from time import sleep

# This cell enables inline plotting in the notebook
%matplotlib inline

import matplotlib
import numpy as np
import matplotlib.pyplot as plt
loadvar()
!auth-tokens-refresh


/home/jovyan/agave
Requirement already satisfied: setvar in /opt/conda/lib/python3.6/site-packages
CMD=jobs-output-get 8979877899256983065-242ac11b-0001-007 fork-command-1.out
INPUTS={"datafile":"agave://nectar-storage-stevenrbrandt/funwave-tvd-docker/rundock.tgz"}
JOB_FILE=job-remote-13262.txt
JOB_ID=8979877899256983065-242ac11b-0001-007
OUTPUT=Successfully submitted job 8979877899256983065-242ac11b-0001-007
REMOTE_COMMAND=tar xzf rundock.tgz && bash rundock.sh
REQUESTBIN_URL=https://requestbin.agaveapi.co/pkxilxpk
STAT=FINISHED
VM_IPADDRESS=18.216.53.253
Token for agave.prod:stevenrbrandt successfully refreshed and cached for 14400 seconds
a283883161e6f7fb10274d6a1921d153

Creating the Docker Image

To start, we need a Dockerfile, which has a number of simple commands. It starts with "FROM" which can specify any docker image available from DockerHub. That not only includes basic operating systems such as "ubunto", "fedora", "centos", etc. but specialized containers made by anyone with a dockerhub account. I've provided "science-base" which has OpenMPI 2.1.1 and some standard compilers, i.e. gfortran, gcc, and g++.

MAINTAINER is a bit of metadata that (hopefully) will allow you to contact the container's creator, if need be.

WORKDIR is the dockerfile equivalent of the "cd" command. Note that running "cd" will not change your directory.

RUN simply runs the command that follows. Because the container is saved after each step, we want to avoid creating files that we don't want to keep (we want containers to be as small as possible).

USER specifies the user id for running subsequent RUN commands.

COPY can be used to copy files into the container from the build directory.

ENTRYPOINT is a script that runs when the container starts up. What our script does is create a new user on the docker image with a user id and name that is convenient.


In [2]:
writefile("funwave-tvd-docker/Dockerfile","""
FROM stevenrbrandt/science-base
USER root
RUN mkdir -p /home/install
RUN chown jovyan /home/install
USER jovyan

MAINTAINER Steven R. Brandt <sbrandt@cct.lsu.edu>
RUN cd /home/install && \
    git clone https://github.com/fengyanshi/FUNWAVE-TVD && \
    cd FUNWAVE-TVD/src && \
    perl -p -i -e 's/FLAG_8 = -DCOUPLING/#$&/' Makefile && \
    make

WORKDIR /home/install/FUNWAVE-TVD/src
RUN mkdir -p /home/jovyan/rundir
WORKDIR /home/jovyan/rundir
""")


Writing file `funwave-tvd-docker/Dockerfile'

Now that we've create our Dockerfile and entrypoint.sh, bundle them up in a tarball and send them somewhere that agave can access them.


In [3]:
!tar -czf dockerjob.tgz -C funwave-tvd-docker Dockerfile
!files-mkdir -S ${AGAVE_STORAGE_SYSTEM_ID} -N funwave-tvd-docker
!files-upload -F dockerjob.tgz -S ${AGAVE_STORAGE_SYSTEM_ID} funwave-tvd-docker/


Successfully created folder funwave-tvd-docker
Uploading dockerjob.tgz...
######################################################################## 100.0%

In [4]:
import runagavecmd as r
import imp
imp.reload(r)


Out[4]:
<module 'runagavecmd' from '/home/jovyan/agave/runagavecmd.py'>

Run the docker build command. We will "tag" this build with the name "funwave-tvd" when it is complete.


In [5]:
r.runagavecmd(
    "tar xzf dockerjob.tgz && sudo docker build --rm -t funwave-tvd-2 .",
    "agave://${AGAVE_STORAGE_SYSTEM_ID}/funwave-tvd-docker/dockerjob.tgz"
)


REMOTE_COMMAND=tar xzf dockerjob.tgz && sudo docker build --rm -t funwave-tvd-2 .
REQUESTBIN_URL=https://requestbin.agaveapi.co/qyc1k8qy

 ** QUERY STRING FOR REQUESTBIN **
https://requestbin.agaveapi.co/qyc1k8qy?inspect

INPUTS={"datafile":"agave://nectar-storage-stevenrbrandt/funwave-tvd-docker/dockerjob.tgz"}
JOB_FILE=job-remote-19597.txt
Writing file `job-remote-19597.txt'
OUTPUT=Successfully submitted job 4040708458529943065-242ac11b-0001-007
JOB_ID=4040708458529943065-242ac11b-0001-007
STAT=PENDING
STAT=PENDING
STAT=PENDING
STAT=PROCESSING_INPUTS
STAT=STAGING_INPUTS
STAT=STAGED
STAT=STAGED
STAT=SUBMITTING
STAT=SUBMITTING
STAT=SUBMITTING
STAT=FINISHED
CMD=jobs-output-get 4040708458529943065-242ac11b-0001-007 fork-command-1.out
All done! Output follows.
Reading file `fork-command-1.out'
======================================================================
Sending build context to Docker daemon  12.29kB

Step 1/10 : FROM stevenrbrandt/science-base
 ---> e93db47f971c
Step 2/10 : USER root
 ---> Using cache
 ---> b94fc308f479
Step 3/10 : RUN mkdir -p /home/install
 ---> Using cache
 ---> beccae742726
Step 4/10 : RUN chown jovyan /home/install
 ---> Using cache
 ---> 81c2bd496517
Step 5/10 : USER jovyan
 ---> Using cache
 ---> c16d5c774336
Step 6/10 : MAINTAINER Steven R. Brandt <sbrandt@cct.lsu.edu>
 ---> Using cache
 ---> f992707e0150
Step 7/10 : RUN cd /home/install &&     git clone https://github.com/fengyanshi/FUNWAVE-TVD &&     cd FUNWAVE-TVD/src &&     perl -p -i -e 's/FLAG_8 = -DCOUPLING/#$&/' Makefile &&     make
 ---> Using cache
 ---> 04bd5f3529fb
Step 8/10 : WORKDIR /home/install/FUNWAVE-TVD/src
 ---> Using cache
 ---> 396b8c1fb201
Step 9/10 : RUN mkdir -p /home/jovyan/rundir
 ---> Using cache
 ---> 3aecce97d0e7
Step 10/10 : WORKDIR /home/jovyan/rundir
 ---> Using cache
 ---> 22beb9c32c31
Successfully built 22beb9c32c31
Successfully tagged funwave-tvd-2:latest


In [6]:
!jobs-output-get ${JOB_ID} fork-command-1.err
!cat fork-command-1.err



Running the Docker Image

It is possible to run docker interactively, but that isn't convenient inside scripts. So instead, we start it in detached mode, with the -d flag.

Because your docker image has its own internal file system, it can't see files on the host machine. You can, however, transfer them using the "docker cp" command.

Running docker is slightly tricky. When a Docker image starts up, you can execute any command you want--but when you type "exit" all the changes you've made to the file system vanish. Therefore it's necessary to copy them out before the docker container stops.


In [7]:
writefile("rundock.sh","""
rm -fr cid.txt out.tgz

# Start a docker image running in detached mode, write the container id to cid.txt
sudo docker run -d -it --rm --cidfile cid.txt funwave-tvd-2 bash

# Store the container id in CID for convenience
CID=\$(cat cid.txt)

# Copy the input.txt file into the running image
sudo docker cp input.txt \$CID:/home/jovyan/rundir/

# Run funwave on the image
sudo docker exec --user jovyan \$CID mpirun -np 2 /home/install/FUNWAVE-TVD/src/funwave_vessel

# Extract the output files from the running image
# Having them in a tgz makes it more convenient to fetch them with jobs-output-get
sudo docker exec --user jovyan \$CID tar czf - output > out.tgz

# Stop the image
sudo docker stop \$CID

# List the output files
tar tzf out.tgz
""")


Writing file `rundock.sh'

Upload the input.txt file and the rundock.sh script.


In [8]:
!tar czf rundock.tgz rundock.sh input.txt
!files-upload -F rundock.tgz -S ${AGAVE_STORAGE_SYSTEM_ID} funwave-tvd-docker/


Uploading rundock.tgz...
######################################################################## 100.0%

Execute the rundock.sh script


In [9]:
r.runagavecmd(
    "tar xzf rundock.tgz && bash rundock.sh",
    "agave://${AGAVE_STORAGE_SYSTEM_ID}/funwave-tvd-docker/rundock.tgz")


REMOTE_COMMAND=tar xzf rundock.tgz && bash rundock.sh
REQUESTBIN_URL=https://requestbin.agaveapi.co/1bl4xtf1

 ** QUERY STRING FOR REQUESTBIN **
https://requestbin.agaveapi.co/1bl4xtf1?inspect

INPUTS={"datafile":"agave://nectar-storage-stevenrbrandt/funwave-tvd-docker/rundock.tgz"}
JOB_FILE=job-remote-19597.txt
Writing file `job-remote-19597.txt'
OUTPUT=Successfully submitted job 799296640238743065-242ac11b-0001-007
JOB_ID=799296640238743065-242ac11b-0001-007
STAT=PENDING
STAT=PENDING
STAT=PENDING
STAT=STAGING_INPUTS
STAT=STAGED
STAT=STAGED
STAT=STAGED
STAT=SUBMITTING
STAT=SUBMITTING
STAT=SUBMITTING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=FINISHED
CMD=jobs-output-get 799296640238743065-242ac11b-0001-007 fork-command-1.out
All done! Output follows.
Reading file `fork-command-1.out'
======================================================================
10a6e2bb73f5dc8718306f090faaebe814345abee91ff763773ab127f4049798
 BATHY_CORRECTION DOES NOT EXIST. USE DEFAULT VALUE
 INI_UVZ DOES NOT EXIST. USE DEFAULT VALUE
 WaterLevel DOES NOT EXIST. USE DEFAULT VALUE
 ETA_LIMITER DOES NOT EXIST. USE DEFAULT VALUE
 EqualEnergy DOES NOT EXIST. USE DEFAULT VALUE
 PERIODIC DOES NOT EXIST. USE DEFAULT VALUE
 DIFFUSION_SPONGE DOES NOT EXIST. USE DEFAULT VALUE
 DIRECT_SPONGE DOES NOT EXIST. USE DEFAULT VALUE
 FRICTION_SPONGE DOES NOT EXIST. USE DEFAULT VALUE
 OBSTACLE_FILE DOES NOT EXIST. USE DEFAULT VALUE
 BREAKWATER_FILE DOES NOT EXIST. USE DEFAULT VALUE
 DISPERSION DOES NOT EXIST. USE DEFAULT VALUE
         DISPERSION Default:  DISPERSION
 Gamma1 DOES NOT EXIST. USE DEFAULT VALUE
        Gamma1 Default:  1.0: DISPERSION
 Gamma2 DOES NOT EXIST. USE DEFAULT VALUE
              Gamma2 Default:  1.0: Full nonlinear
 Beta_ref DOES NOT EXIST. USE DEFAULT VALUE
               Beta_ref Default:  -0.531
 Gamma3 DOES NOT EXIST. USE DEFAULT VALUE
                      Gamma3 Default:  1.0: NOT fully linear
 VISCOSITY_BREAKING DOES NOT EXIST. USE DEFAULT VALUE
                   VISCOSITY_BREAKING Default:  SWE Breaking
 SWE_ETA_DEP DOES NOT EXIST. USE DEFAULT VALUE
               SWE_ETA_DEP Default:  0.8
 Friction_Matrix DOES NOT EXIST. USE DEFAULT VALUE
             Friction_Matrix Default:  constant Cd
 Cd DOES NOT EXIST. USE DEFAULT VALUE
                  Cd_fixed Default:  0.0
 Time_Scheme DOES NOT EXIST. USE DEFAULT VALUE
       Time_Scheme Default:  Runge_Kutta
 CONSTRUCTION DOES NOT EXIST. USE DEFAULT VALUE
 HIGH_ORDER DOES NOT EXIST. USE DEFAULT VALUE
  HIGH_ORDER                     NOT DEFINED, USE FOURTH-ORDER
 CFL DOES NOT EXIST. USE DEFAULT VALUE
                       CFL Default:  0.5
 DT_fixed DOES NOT EXIST. USE DEFAULT VALUE
 FroudeCap DOES NOT EXIST. USE DEFAULT VALUE
                 FroudeCap Default:  3.0
 MinDepth DOES NOT EXIST. USE DEFAULT VALUE
                MinDepth Default:  0.1 m
 MinDepthFrc DOES NOT EXIST. USE DEFAULT VALUE
             MinDepthFrc Default:  0.1 m
 SHOW_BREAKING DOES NOT EXIST. USE DEFAULT VALUE
            SHOW_BREAKING Default:  TRUE
 Cbrk1 DOES NOT EXIST. USE DEFAULT VALUE
                    Cbrk1 Default:  0.65
 Cbrk2 DOES NOT EXIST. USE DEFAULT VALUE
                    Cbrk2 Default:  0.35
 WAVEMAKER_Cbrk DOES NOT EXIST. USE DEFAULT VALUE
            WAVEMAKER_Cbrk Default:  1.0
 WAVEMAKER_VIS DOES NOT EXIST. USE DEFAULT VALUE
           WAVEMAKER_VIS Default:  FALSE
 T_INTV_mean DOES NOT EXIST. USE DEFAULT VALUE
             T_INTV_mean Default:  LARGE
 STEADY_TIME DOES NOT EXIST. USE DEFAULT VALUE
             STEADY_TIME Default:  LARGE
 C_smg DOES NOT EXIST. USE DEFAULT VALUE
                     C_smg Default:  0.0
 nu_bkg DOES NOT EXIST. USE DEFAULT VALUE
                    nu_bkg Default:  0.0
 FIELD_IO_TYPE DOES NOT EXIST. USE DEFAULT VALUE
 NumberStations DOES NOT EXIST. USE DEFAULT VALUE
 OUTPUT_RES DOES NOT EXIST. USE DEFAULT VALUE
         OUTPUT_RES NOT FOUND, OUTPUT_RES=1: full resolution
 DEPTH_OUT DOES NOT EXIST. USE DEFAULT VALUE
 Hmax DOES NOT EXIST. USE DEFAULT VALUE
 Hmin DOES NOT EXIST. USE DEFAULT VALUE
 Umax DOES NOT EXIST. USE DEFAULT VALUE
 MFmax DOES NOT EXIST. USE DEFAULT VALUE
 VORmax DOES NOT EXIST. USE DEFAULT VALUE
 MASK DOES NOT EXIST. USE DEFAULT VALUE
 MASK9 DOES NOT EXIST. USE DEFAULT VALUE
 Umean DOES NOT EXIST. USE DEFAULT VALUE
 Vmean DOES NOT EXIST. USE DEFAULT VALUE
 ETAmean DOES NOT EXIST. USE DEFAULT VALUE
 WaveHeight DOES NOT EXIST. USE DEFAULT VALUE
 SXL DOES NOT EXIST. USE DEFAULT VALUE
 SXR DOES NOT EXIST. USE DEFAULT VALUE
 SYL DOES NOT EXIST. USE DEFAULT VALUE
 SYR DOES NOT EXIST. USE DEFAULT VALUE
 SourceX DOES NOT EXIST. USE DEFAULT VALUE
 SourceY DOES NOT EXIST. USE DEFAULT VALUE
 P DOES NOT EXIST. USE DEFAULT VALUE
 Q DOES NOT EXIST. USE DEFAULT VALUE
 Fx DOES NOT EXIST. USE DEFAULT VALUE
 Fy DOES NOT EXIST. USE DEFAULT VALUE
 Gx DOES NOT EXIST. USE DEFAULT VALUE
 Gy DOES NOT EXIST. USE DEFAULT VALUE
 AGE DOES NOT EXIST. USE DEFAULT VALUE
 OUT_NU DOES NOT EXIST. USE DEFAULT VALUE
 TMP DOES NOT EXIST. USE DEFAULT VALUE
      EtaBlowVal Default:  100xmax_depth
 ----------------- STATISTICS ----------------
  TIME        DT
  0.4428E-01  0.4428E-01
  MassVolume  Energy      MaxEta      MinEta      Max U       Max V 
  0.1937E+04  0.1247E+08  0.2996E+01 -0.4596E-06  0.4050E-01  0.4050E-01
  MaxTotalU   PhaseS      Froude 
  0.4050E-01  0.1066E+02  0.3800E-02
   PRINTING FILE NO.     1  TIME/TOTAL:        0.044 /       3.000
 ----------------- STATISTICS ----------------
  TIME        DT
  0.1017E+01  0.4414E-01
  MassVolume  Energy      MaxEta      MinEta      Max U       Max V 
  0.1927E+04  0.1246E+08  0.1900E+01 -0.1727E-05  0.7329E+00  0.7325E+00
  MaxTotalU   PhaseS      Froude 
  0.7338E+00  0.1050E+02  0.6987E-01
   PRINTING FILE NO.     2  TIME/TOTAL:        1.017 /       3.000
 ----------------- STATISTICS ----------------
  TIME        DT
  0.2037E+01  0.4473E-01
  MassVolume  Energy      MaxEta      MinEta      Max U       Max V 
  0.1906E+04  0.1246E+08  0.7393E+00 -0.2109E-05  0.9186E+00  0.9148E+00
  MaxTotalU   PhaseS      Froude 
  0.9215E+00  0.1024E+02  0.8998E-01
   PRINTING FILE NO.     3  TIME/TOTAL:        2.037 /       3.000
 ----------------- STATISTICS ----------------
  TIME        DT
  0.3032E+01  0.4566E-01
  MassVolume  Energy      MaxEta      MinEta      Max U       Max V 
  0.1890E+04  0.1246E+08  0.7188E+00 -0.9413E+00  0.7352E+00  0.6932E+00
  MaxTotalU   PhaseS      Froude 
  0.7378E+00  0.1019E+02  0.7239E-01
   PRINTING FILE NO.     4  TIME/TOTAL:        3.032 /       3.000
 Simulation takes   17.199041255997145      seconds
 Normal Termination!
10a6e2bb73f5dc8718306f090faaebe814345abee91ff763773ab127f4049798
output/
output/eta_00001
output/v_00003
output/u_00001
output/eta_00003
output/v_00001
output/u_00002
output/eta_00004
output/eta_00002
output/u_00003
output/v_00004
output/v_00002
output/u_00004

Get the output of the job back to our local machine


In [10]:
!jobs-output-list ${JOB_ID}
!jobs-output-get ${JOB_ID} out.tgz
!tar xzf out.tgz


.agave.archive
.agave.log
cid.txt
fork-command-1.err
fork-command-1.ipcexe
fork-command-1.out
fork-command-1.pid
fork-test.txt
fork-wrapper.txt
input.txt
out.tgz
rundock.sh
rundock.tgz
######################################################################## 100.0%                       66.2%

In [11]:
!head output/eta_00010


head: cannot open 'output/eta_00010' for reading: No such file or directory

Running with Singularity

If we have a public docker image, we can run it directly with Singularity. Singularity is desiged to be more HPC friendly than Docker. First, because it doesn't all the running user to access any user id but their own inside the container, and second, because singularity images can be run through MPI, making it easier to scale up to a distributed cluser.

In this first step, we build the singularity installation. Because the result of this job is intended to be an installation for subsequent jobs, we install it to a hard-coded directory rather than using the normal Agave job directory.


In [12]:
!files-mkdir -S ${AGAVE_STORAGE_SYSTEM_ID} -N sing
!files-upload -F input.txt -S ${AGAVE_STORAGE_SYSTEM_ID} sing/
r.runagavecmd(
            "mkdir -p ~/singu && "+
            "cd ~/singu && "+
            "rm -f funwave-tvd.img && "+
            "singularity create funwave-tvd.img --size 2000 && "+
            "singularity import funwave-tvd.img docker://stevenrbrandt/funwave-tvd-2:latest")


Successfully created folder sing
Uploading input.txt...
######################################################################## 100.0%
REMOTE_COMMAND=mkdir -p ~/singu && cd ~/singu && rm -f funwave-tvd.img && singularity create funwave-tvd.img --size 2000 && singularity import funwave-tvd.img docker://stevenrbrandt/funwave-tvd-2:latest
REQUESTBIN_URL=https://requestbin.agaveapi.co/yj6r3eyj

 ** QUERY STRING FOR REQUESTBIN **
https://requestbin.agaveapi.co/yj6r3eyj?inspect

INPUTS={}
JOB_FILE=job-remote-19597.txt
Writing file `job-remote-19597.txt'
OUTPUT=Successfully submitted job 4296939754503082471-242ac11b-0001-007
JOB_ID=4296939754503082471-242ac11b-0001-007
STAT=PENDING
STAT=PENDING
STAT=PENDING
STAT=STAGED
STAT=STAGED
STAT=STAGED
STAT=SUBMITTING
STAT=SUBMITTING
STAT=SUBMITTING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
CMD=jobs-output-get 4296939754503082471-242ac11b-0001-007 fork-command-1.out
All done! Output follows.
Reading file `fork-command-1.out'
======================================================================
Initializing Singularity image subsystem
Opening image file: funwave-tvd.img
Creating 768MiB image
Binding image to loop
Creating file system within image
Image is done: funwave-tvd.img
Docker image path: index.docker.io/stevenrbrandt/funwave-tvd-2:latest
Cache folder set to /home/jovyan/.singularity/docker

[1/13] ||----------------------------------|   0.0% 
[1/13] |==|--------------------------------|   7.7% 
[2/13] |=====|-----------------------------|  15.4% 
[3/13] |========|--------------------------|  23.1% 
[4/13] |==========|------------------------|  30.8% 
[5/13] |=============|---------------------|  38.5% 
[6/13] |================|------------------|  46.2% 
[7/13] |==================|----------------|  53.8% 
[8/13] |=====================|-------------|  61.5% 
[9/13] |========================|----------|  69.2% 
[10/13] |==========================|--------|  76.9% 
[11/13] |=============================|-----|  84.6% 
[12/13] |================================|--|  92.3% 
[13/13] |===================================| 100.0% 
Importing: base Singularity environment
Importing: /home/jovyan/.singularity/docker/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4.tar.gz
Importing: /home/jovyan/.singularity/docker/sha256:be96bf654103f9670434de9ccc428ee55fb1f14531619dd6cb8fbd85b5765393.tar.gz
Importing: /home/jovyan/.singularity/docker/sha256:3f38f85a45654716a81c8ad5c89a0cfc820932422078150624f1f83661b6fec7.tar.gz
Importing: /home/jovyan/.singularity/docker/sha256:15bd671928b4d58adb14870cf74c013f9781d4979c944d851827afde27d5bdb8.tar.gz
Importing: /home/jovyan/.singularity/docker/sha256:203b799f83cfd0d284472fdd78efc556302c353e270b9974d03cec71a2faf14b.tar.gz
Importing: /home/jovyan/.singularity/docker/sha256:c20f403f88792630033b7760769f60fe2da3271d0fdecce4eeeed17c5adfaf21.tar.gz
Importing: /home/jovyan/.singularity/docker/sha256:c912233ab5d5939d997ab7a9110bda1f64e2d3cdf3808a21d95736b1da58766f.tar.gz
Importing: /home/jovyan/.singularity/docker/sha256:022aa0a604e76c9b9223aa277639f27104b61b6e1c1428ed4174d4c133cd685c.tar.gz
Importing: /home/jovyan/.singularity/docker/sha256:630ceed02486a128742ac2b5c3105c88ad28f410a0b2e08ff4ddae2878483924.tar.gz
Importing: /home/jovyan/.singularity/docker/sha256:25bb6f291ceb2dcdd3e24aa3a00ef625e6e1a022779283fc2c1b2ccecbc0d162.tar.gz
Importing: /home/jovyan/.singularity/docker/sha256:a7cad26d035791e5bf04d6479b03dee59d40a504cfd0b54840966f98b39abfc4.tar.gz
Importing: /home/jovyan/.singularity/docker/sha256:f7d512d8250289b154a8e982ff1272ed1376e407bf21f314cd55627c22e5598e.tar.gz
Importing: /home/jovyan/.singularity/docker/sha256:60730f9603637b2b4d71f56eca6b7cf26f3c3a59478c45f1e53733901ea89b02.tar.gz

Now that the Singularity image is built, we can run it with mpi. Notice that mpi executes the singularity command. The tricky part here is to make sure you've got the same version of mpi running inside and outside the container.


In [13]:
!files-upload -F input.txt -S ${AGAVE_STORAGE_SYSTEM_ID} ./
r.runagavecmd(
    "export LD_LIBRARY_PATH=/usr/local/lib && "+
    "mpirun -np 2 singularity exec ~/singu/funwave-tvd.img /home/install/FUNWAVE-TVD/src/funwave_vessel && "+
    "tar cvzf singout.tgz output",
    "agave://${AGAVE_STORAGE_SYSTEM_ID}/input.txt"
)


Uploading input.txt...
######################################################################## 100.0%
REMOTE_COMMAND=export LD_LIBRARY_PATH=/usr/local/lib && mpirun -np 2 singularity exec ~/singu/funwave-tvd.img /home/install/FUNWAVE-TVD/src/funwave_vessel && tar cvzf singout.tgz output
REQUESTBIN_URL=https://requestbin.agaveapi.co/17u7mgg1

 ** QUERY STRING FOR REQUESTBIN **
https://requestbin.agaveapi.co/17u7mgg1?inspect

INPUTS={"datafile":"agave://nectar-storage-stevenrbrandt/input.txt"}
JOB_FILE=job-remote-19597.txt
Writing file `job-remote-19597.txt'
OUTPUT=Successfully submitted job 5360771817640956391-242ac11b-0001-007
JOB_ID=5360771817640956391-242ac11b-0001-007
STAT=PENDING
STAT=PENDING
STAT=PENDING
STAT=PROCESSING_INPUTS
STAT=STAGING_INPUTS
STAT=STAGED
STAT=STAGED
STAT=SUBMITTING
STAT=SUBMITTING
STAT=SUBMITTING
STAT=SUBMITTING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=CLEANING_UP
STAT=FINISHED
CMD=jobs-output-get 5360771817640956391-242ac11b-0001-007 fork-command-1.out
All done! Output follows.
Reading file `fork-command-1.out'
======================================================================
 BATHY_CORRECTION DOES NOT EXIST. USE DEFAULT VALUE
 INI_UVZ DOES NOT EXIST. USE DEFAULT VALUE
 WaterLevel DOES NOT EXIST. USE DEFAULT VALUE
 ETA_LIMITER DOES NOT EXIST. USE DEFAULT VALUE
 EqualEnergy DOES NOT EXIST. USE DEFAULT VALUE
 PERIODIC DOES NOT EXIST. USE DEFAULT VALUE
 DIFFUSION_SPONGE DOES NOT EXIST. USE DEFAULT VALUE
 DIRECT_SPONGE DOES NOT EXIST. USE DEFAULT VALUE
 FRICTION_SPONGE DOES NOT EXIST. USE DEFAULT VALUE
 OBSTACLE_FILE DOES NOT EXIST. USE DEFAULT VALUE
 BREAKWATER_FILE DOES NOT EXIST. USE DEFAULT VALUE
 DISPERSION DOES NOT EXIST. USE DEFAULT VALUE
         DISPERSION Default:  DISPERSION
 Gamma1 DOES NOT EXIST. USE DEFAULT VALUE
        Gamma1 Default:  1.0: DISPERSION
 Gamma2 DOES NOT EXIST. USE DEFAULT VALUE
              Gamma2 Default:  1.0: Full nonlinear
 Beta_ref DOES NOT EXIST. USE DEFAULT VALUE
               Beta_ref Default:  -0.531
 Gamma3 DOES NOT EXIST. USE DEFAULT VALUE
                      Gamma3 Default:  1.0: NOT fully linear
 VISCOSITY_BREAKING DOES NOT EXIST. USE DEFAULT VALUE
                   VISCOSITY_BREAKING Default:  SWE Breaking
 SWE_ETA_DEP DOES NOT EXIST. USE DEFAULT VALUE
               SWE_ETA_DEP Default:  0.8
 Friction_Matrix DOES NOT EXIST. USE DEFAULT VALUE
             Friction_Matrix Default:  constant Cd
 Cd DOES NOT EXIST. USE DEFAULT VALUE
                  Cd_fixed Default:  0.0
 Time_Scheme DOES NOT EXIST. USE DEFAULT VALUE
       Time_Scheme Default:  Runge_Kutta
 CONSTRUCTION DOES NOT EXIST. USE DEFAULT VALUE
 HIGH_ORDER DOES NOT EXIST. USE DEFAULT VALUE
  HIGH_ORDER                     NOT DEFINED, USE FOURTH-ORDER
 CFL DOES NOT EXIST. USE DEFAULT VALUE
                       CFL Default:  0.5
 DT_fixed DOES NOT EXIST. USE DEFAULT VALUE
 FroudeCap DOES NOT EXIST. USE DEFAULT VALUE
                 FroudeCap Default:  3.0
 MinDepth DOES NOT EXIST. USE DEFAULT VALUE
                MinDepth Default:  0.1 m
 MinDepthFrc DOES NOT EXIST. USE DEFAULT VALUE
             MinDepthFrc Default:  0.1 m
 SHOW_BREAKING DOES NOT EXIST. USE DEFAULT VALUE
            SHOW_BREAKING Default:  TRUE
 Cbrk1 DOES NOT EXIST. USE DEFAULT VALUE
                    Cbrk1 Default:  0.65
 Cbrk2 DOES NOT EXIST. USE DEFAULT VALUE
                    Cbrk2 Default:  0.35
 WAVEMAKER_Cbrk DOES NOT EXIST. USE DEFAULT VALUE
            WAVEMAKER_Cbrk Default:  1.0
 WAVEMAKER_VIS DOES NOT EXIST. USE DEFAULT VALUE
           WAVEMAKER_VIS Default:  FALSE
 T_INTV_mean DOES NOT EXIST. USE DEFAULT VALUE
             T_INTV_mean Default:  LARGE
 STEADY_TIME DOES NOT EXIST. USE DEFAULT VALUE
             STEADY_TIME Default:  LARGE
 C_smg DOES NOT EXIST. USE DEFAULT VALUE
                     C_smg Default:  0.0
 nu_bkg DOES NOT EXIST. USE DEFAULT VALUE
                    nu_bkg Default:  0.0
 FIELD_IO_TYPE DOES NOT EXIST. USE DEFAULT VALUE
 NumberStations DOES NOT EXIST. USE DEFAULT VALUE
 OUTPUT_RES DOES NOT EXIST. USE DEFAULT VALUE
         OUTPUT_RES NOT FOUND, OUTPUT_RES=1: full resolution
 DEPTH_OUT DOES NOT EXIST. USE DEFAULT VALUE
 Hmax DOES NOT EXIST. USE DEFAULT VALUE
 Hmin DOES NOT EXIST. USE DEFAULT VALUE
 Umax DOES NOT EXIST. USE DEFAULT VALUE
 MFmax DOES NOT EXIST. USE DEFAULT VALUE
 VORmax DOES NOT EXIST. USE DEFAULT VALUE
 MASK DOES NOT EXIST. USE DEFAULT VALUE
 MASK9 DOES NOT EXIST. USE DEFAULT VALUE
 Umean DOES NOT EXIST. USE DEFAULT VALUE
 Vmean DOES NOT EXIST. USE DEFAULT VALUE
 ETAmean DOES NOT EXIST. USE DEFAULT VALUE
 WaveHeight DOES NOT EXIST. USE DEFAULT VALUE
 SXL DOES NOT EXIST. USE DEFAULT VALUE
 SXR DOES NOT EXIST. USE DEFAULT VALUE
 SYL DOES NOT EXIST. USE DEFAULT VALUE
 SYR DOES NOT EXIST. USE DEFAULT VALUE
 SourceX DOES NOT EXIST. USE DEFAULT VALUE
 SourceY DOES NOT EXIST. USE DEFAULT VALUE
 P DOES NOT EXIST. USE DEFAULT VALUE
 Q DOES NOT EXIST. USE DEFAULT VALUE
 Fx DOES NOT EXIST. USE DEFAULT VALUE
 Fy DOES NOT EXIST. USE DEFAULT VALUE
 Gx DOES NOT EXIST. USE DEFAULT VALUE
 Gy DOES NOT EXIST. USE DEFAULT VALUE
 AGE DOES NOT EXIST. USE DEFAULT VALUE
 OUT_NU DOES NOT EXIST. USE DEFAULT VALUE
 TMP DOES NOT EXIST. USE DEFAULT VALUE
      EtaBlowVal Default:  100xmax_depth
 ----------------- STATISTICS ----------------
  TIME        DT
  0.4428E-01  0.4428E-01
  MassVolume  Energy      MaxEta      MinEta      Max U       Max V 
  0.1937E+04  0.1247E+08  0.2996E+01 -0.4596E-06  0.4050E-01  0.4050E-01
  MaxTotalU   PhaseS      Froude 
  0.4050E-01  0.1066E+02  0.3800E-02
   PRINTING FILE NO.     1  TIME/TOTAL:        0.044 /       3.000
 ----------------- STATISTICS ----------------
  TIME        DT
  0.1017E+01  0.4414E-01
  MassVolume  Energy      MaxEta      MinEta      Max U       Max V 
  0.1927E+04  0.1246E+08  0.1900E+01 -0.1727E-05  0.7329E+00  0.7325E+00
  MaxTotalU   PhaseS      Froude 
  0.7338E+00  0.1050E+02  0.6987E-01
   PRINTING FILE NO.     2  TIME/TOTAL:        1.017 /       3.000
 ----------------- STATISTICS ----------------
  TIME        DT
  0.2037E+01  0.4473E-01
  MassVolume  Energy      MaxEta      MinEta      Max U       Max V 
  0.1906E+04  0.1246E+08  0.7393E+00 -0.2109E-05  0.9186E+00  0.9148E+00
  MaxTotalU   PhaseS      Froude 
  0.9215E+00  0.1024E+02  0.8998E-01
   PRINTING FILE NO.     3  TIME/TOTAL:        2.037 /       3.000
 ----------------- STATISTICS ----------------
  TIME        DT
  0.3032E+01  0.4566E-01
  MassVolume  Energy      MaxEta      MinEta      Max U       Max V 
  0.1890E+04  0.1246E+08  0.7188E+00 -0.9413E+00  0.7352E+00  0.6932E+00
  MaxTotalU   PhaseS      Froude 
  0.7378E+00  0.1019E+02  0.7239E-01
   PRINTING FILE NO.     4  TIME/TOTAL:        3.032 /       3.000
 Simulation takes   15.861142730995198      seconds
 Normal Termination!
output/
output/eta_00001
output/v_00003
output/u_00001
output/eta_00003
output/v_00001
output/u_00002
output/eta_00004
output/eta_00002
output/u_00003
output/v_00004
output/v_00002
output/u_00004


In [14]:
!jobs-output-get ${JOB_ID} singout.tgz
!rm -fr output
!tar xzf singout.tgz


######################################################################## 100.0%

In [20]:
!head output/v_00003


   -0.370381E-07   -0.264216E-07   -0.453174E-07   -0.615901E-07   -0.867047E-07   -0.916501E-07   -0.873622E-07   -0.295986E-07    0.365446E-08   -0.118685E-07   -0.359654E-07   -0.657907E-07   -0.491518E-07   -0.167981E-07   -0.261493E-07   -0.331495E-07   -0.531462E-07   -0.380234E-07   -0.715021E-08   -0.531698E-07   -0.288188E-07   -0.224122E-07   -0.117357E-07   -0.334322E-07   -0.607799E-07   -0.793287E-07   -0.801748E-07   -0.278102E-07    0.820207E-08   -0.176744E-07   -0.376689E-07   -0.505262E-07   -0.474473E-07   -0.120814E-07    0.521564E-08   -0.202734E-07   -0.502107E-07   -0.495059E-07   -0.170843E-07   -0.794791E-09    0.287557E-07    0.286107E-08   -0.407979E-07   -0.571593E-07   -0.166976E-07   -0.917486E-08   -0.159806E-08   -0.293239E-07   -0.487221E-07   -0.363046E-07   -0.312466E-07   -0.454842E-07   -0.737199E-07   -0.372320E-07   -0.263822E-07   -0.606216E-07   -0.521675E-07   -0.414295E-08   -0.285707E-08   -0.387530E-07   -0.759180E-07   -0.594871E-07   -0.381240E-07   -0.243736E-07   -0.351791E-07   -0.454820E-07   -0.330221E-07   -0.431807E-07   -0.492954E-07   -0.323381E-07   -0.366857E-07   -0.224607E-07    0.536307E-08   -0.184430E-07   -0.379402E-07   -0.367824E-07   -0.425240E-07   -0.580965E-07   -0.256130E-07   -0.361566E-07    0.143026E-07   -0.978389E-08   -0.586753E-07   -0.855737E-07   -0.700330E-07   -0.243344E-07   -0.316075E-07   -0.630901E-08    0.129229E-07   -0.638592E-08   -0.243215E-07   -0.272797E-07   -0.252633E-07   -0.508527E-08   -0.886411E-09   -0.405799E-07   -0.709902E-07   -0.646340E-07   -0.539962E-07   -0.289094E-07   -0.267936E-07   -0.608653E-07   -0.683233E-07   -0.270277E-07   -0.373515E-07   -0.293033E-07   -0.103415E-07   -0.379871E-07   -0.164734E-07   -0.107011E-07   -0.271246E-07   -0.197447E-07   -0.627131E-07   -0.101714E-06   -0.107446E-06   -0.865976E-07   -0.311182E-07   -0.163863E-07   -0.350941E-07   -0.514499E-07   -0.847307E-07   -0.991687E-07   -0.107440E-06   -0.122014E-06   -0.114511E-06   -0.866306E-07   -0.825142E-07   -0.107459E-06   -0.113634E-06   -0.118026E-06   -0.150423E-06   -0.185461E-06   -0.204589E-06   -0.216434E-06   -0.267158E-06   -0.300989E-06   -0.346729E-06   -0.403227E-06   -0.468551E-06   -0.555254E-06   -0.634795E-06   -0.711520E-06   -0.806948E-06   -0.944412E-06   -0.107287E-05   -0.122608E-05   -0.140268E-05   -0.162333E-05   -0.185759E-05   -0.212830E-05   -0.244532E-05   -0.280490E-05   -0.321622E-05   -0.368944E-05   -0.421665E-05   -0.482467E-05   -0.552103E-05   -0.631197E-05   -0.721552E-05   -0.824129E-05   -0.940552E-05   -0.107353E-04   -0.122488E-04   -0.139686E-04   -0.159261E-04   -0.181496E-04   -0.206773E-04   -0.235424E-04   -0.267869E-04   -0.304680E-04   -0.346422E-04   -0.393683E-04   -0.447096E-04   -0.507457E-04   -0.575648E-04   -0.652570E-04   -0.739240E-04   -0.836955E-04   -0.946992E-04   -0.107069E-03   -0.120968E-03   -0.136570E-03   -0.154071E-03   -0.173679E-03   -0.195627E-03   -0.220168E-03   -0.247578E-03   -0.278161E-03   -0.312244E-03   -0.350183E-03   -0.392363E-03   -0.439197E-03   -0.491132E-03   -0.548648E-03   -0.612254E-03   -0.682492E-03   -0.759938E-03   -0.845200E-03   -0.938916E-03   -0.104175E-02   -0.115441E-02   -0.127761E-02   -0.141208E-02   -0.155862E-02   -0.171796E-02   -0.189089E-02   -0.207820E-02   -0.228065E-02   -0.249899E-02   -0.273395E-02   -0.298621E-02   -0.325641E-02   -0.354511E-02   -0.385282E-02   -0.417993E-02   -0.452676E-02   -0.489348E-02   -0.528014E-02   -0.568666E-02   -0.611276E-02   -0.655801E-02   -0.702178E-02   -0.750324E-02   -0.800136E-02   -0.851490E-02   -0.904234E-02   -0.958194E-02   -0.101322E-01   -0.106911E-01   -0.112536E-01   -0.118194E-01   -0.123917E-01   -0.129585E-01   -0.135200E-01   -0.140730E-01   -0.146135E-01   -0.151396E-01   -0.156474E-01   -0.161338E-01   -0.165953E-01   -0.170292E-01   -0.174322E-01   -0.178011E-01   -0.181325E-01   -0.184244E-01   -0.186761E-01   -0.188885E-01   -0.190593E-01   -0.191750E-01   -0.192263E-01   -0.192388E-01   -0.192263E-01   -0.191750E-01   -0.190593E-01   -0.188885E-01   -0.186761E-01   -0.184244E-01   -0.181325E-01   -0.178011E-01   -0.174322E-01   -0.170292E-01   -0.165953E-01   -0.161338E-01   -0.156474E-01   -0.151396E-01   -0.146135E-01   -0.140730E-01   -0.135200E-01   -0.129585E-01   -0.123917E-01   -0.118194E-01   -0.112536E-01   -0.106911E-01   -0.101322E-01   -0.958194E-02   -0.904234E-02   -0.851490E-02   -0.800136E-02   -0.750324E-02   -0.702178E-02   -0.655801E-02   -0.611276E-02   -0.568666E-02   -0.528014E-02   -0.489348E-02   -0.452676E-02   -0.417993E-02   -0.385282E-02   -0.354511E-02   -0.325641E-02   -0.298621E-02   -0.273395E-02   -0.249899E-02   -0.228065E-02   -0.207820E-02   -0.189089E-02   -0.171796E-02   -0.155862E-02   -0.141208E-02   -0.127761E-02   -0.115441E-02   -0.104175E-02   -0.938916E-03   -0.845200E-03   -0.759938E-03   -0.682492E-03   -0.612254E-03   -0.548648E-03   -0.491132E-03   -0.439197E-03   -0.392364E-03   -0.350183E-03   -0.312244E-03   -0.278161E-03   -0.247578E-03   -0.220168E-03   -0.195627E-03   -0.173679E-03   -0.154072E-03   -0.136571E-03   -0.120968E-03   -0.107069E-03   -0.946992E-04   -0.836955E-04   -0.739240E-04   -0.652569E-04   -0.575642E-04   -0.507449E-04   -0.447087E-04   -0.393671E-04   -0.346414E-04   -0.304668E-04   -0.267858E-04   -0.235420E-04   -0.206784E-04   -0.181505E-04   -0.159257E-04   -0.139692E-04   -0.122502E-04   -0.107376E-04   -0.940862E-05   -0.824220E-05   -0.721090E-05   -0.630980E-05   -0.552089E-05   -0.482670E-05   -0.421979E-05   -0.368889E-05   -0.322201E-05   -0.281092E-05   -0.245560E-05   -0.213557E-05   -0.186686E-05   -0.162406E-05   -0.140119E-05   -0.123302E-05   -0.108787E-05   -0.949514E-06   -0.812611E-06   -0.699016E-06   -0.633429E-06   -0.567527E-06   -0.484464E-06   -0.407605E-06   -0.339964E-06   -0.312465E-06   -0.281380E-06   -0.220302E-06   -0.190919E-06   -0.157776E-06   -0.143017E-06   -0.158178E-06   -0.164528E-06   -0.131794E-06   -0.101306E-06   -0.867126E-07   -0.995368E-07   -0.114116E-06   -0.118863E-06   -0.141764E-06   -0.130892E-06   -0.800921E-07   -0.537856E-07   -0.361008E-07   -0.806649E-08   -0.429416E-07   -0.782987E-07   -0.731513E-07   -0.457386E-07   -0.272762E-07   -0.199942E-07   -0.289818E-07   -0.153540E-07   -0.192661E-07   -0.755925E-07   -0.521052E-07   -0.275858E-07   -0.355991E-07   -0.519737E-07   -0.440696E-07   -0.208959E-07   -0.441005E-07   -0.183000E-07   -0.556731E-07   -0.713152E-07   -0.383023E-07   -0.265801E-07   -0.324680E-07   -0.457190E-07   -0.307102E-07   -0.169014E-07   -0.129009E-07    0.532344E-08   -0.195307E-07   -0.499705E-07   -0.528522E-07   -0.725266E-07   -0.541334E-07   -0.368727E-07   -0.431557E-07   -0.259984E-07   -0.129517E-07   -0.429034E-07   -0.691278E-07   -0.736828E-07   -0.566911E-07   -0.394391E-07   -0.537709E-07   -0.330791E-07    0.300045E-09    0.661530E-11   -0.478729E-07   -0.783663E-07   -0.821362E-07   -0.828671E-07   -0.951237E-07   -0.401482E-07    0.151596E-07   -0.149809E-07   -0.524474E-07   -0.928579E-07   -0.732621E-07   -0.287197E-07   -0.144439E-08    0.240076E-07   -0.431736E-07   -0.695510E-07   -0.485310E-07   -0.489752E-07   -0.505464E-07   -0.599041E-08   -0.403792E-08   -0.293470E-07   -0.905036E-07   -0.942252E-07   -0.490246E-07   -0.202922E-07   -0.180157E-07   -0.241625E-07   -0.694954E-07   -0.895467E-07   -0.622533E-07   -0.368261E-07   -0.128154E-07   -0.295913E-07   -0.635859E-07   -0.990896E-07   -0.742199E-07   -0.408184E-07   -0.283015E-08    0.945373E-08    0.125971E-07   -0.301275E-07   -0.569471E-07   -0.781747E-07   -0.748916E-07   -0.440774E-07   -0.337795E-07   -0.964221E-08   -0.464799E-07   -0.831190E-07   -0.917154E-07   -0.713160E-07   -0.537729E-07   -0.575811E-07   -0.426448E-07   -0.537945E-07   -0.683450E-07   -0.672369E-07   -0.460913E-07   -0.270848E-07   -0.275067E-07   -0.250059E-07   -0.372548E-07   -0.338110E-07   -0.294900E-07   -0.580005E-07   -0.932209E-07   -0.792152E-07   -0.339107E-07
   -0.774590E-09    0.244346E-07    0.114468E-07   -0.174623E-07   -0.716177E-07   -0.103320E-06   -0.100045E-06    0.118483E-07    0.757278E-07    0.425873E-07   -0.781542E-08   -0.393849E-07   -0.609445E-08    0.301945E-07    0.110521E-07    0.400187E-08   -0.927601E-08    0.108788E-07    0.334078E-07   -0.778556E-08   -0.404745E-08    0.188577E-07    0.467406E-07    0.273384E-07   -0.699353E-08   -0.550968E-07   -0.572451E-07    0.310849E-07    0.841461E-07    0.414149E-07   -0.126095E-08   -0.598285E-08    0.234102E-07    0.776079E-07    0.846676E-07    0.211216E-07   -0.663077E-07   -0.595288E-07    0.154215E-07    0.748034E-07    0.124162E-06    0.687626E-07   -0.711947E-08   -0.326631E-07    0.455169E-07    0.743092E-07    0.592146E-07   -0.555122E-08   -0.299976E-07   -0.326588E-08    0.472432E-07    0.489664E-08   -0.448447E-07   -0.182709E-07    0.160077E-07   -0.630359E-08    0.889716E-08    0.786829E-07    0.845471E-07   -0.776813E-08   -0.723394E-07   -0.667815E-07    0.370151E-08    0.310380E-07    0.163808E-07    0.241722E-07    0.500987E-07    0.100893E-07   -0.224043E-07   -0.388007E-08   -0.977577E-09    0.440141E-07    0.110755E-06    0.753374E-07    0.754767E-08   -0.302793E-07   -0.504430E-07   -0.548098E-07    0.806263E-08    0.469743E-07    0.115147E-06    0.788433E-07   -0.124046E-07   -0.848779E-07   -0.593120E-07    0.168095E-07    0.323456E-07    0.601356E-07    0.803928E-07    0.354741E-07    0.219140E-09    0.202971E-07    0.384538E-07    0.834231E-07    0.757007E-07    0.149302E-07   -0.608033E-07   -0.867807E-07   -0.519805E-07    0.109245E-07    0.537161E-07    0.969121E-08   -0.846035E-08    0.117486E-07   -0.390474E-08    0.899846E-08    0.362640E-07    0.297944E-07    0.496054E-07    0.421846E-07    0.214698E-07    0.156166E-07   -0.617144E-07   -0.119376E-06   -0.114822E-06   -0.541323E-07    0.224453E-07    0.217207E-07   -0.219806E-07   -0.636484E-07   -0.120255E-06   -0.131688E-06   -0.136277E-06   -0.145316E-06   -0.132071E-06   -0.848409E-07   -0.109528E-06   -0.155503E-06   -0.178931E-06   -0.204031E-06   -0.277090E-06   -0.350872E-06   -0.404389E-06   -0.457131E-06   -0.545135E-06   -0.627485E-06   -0.737475E-06   -0.861620E-06   -0.999610E-06   -0.118020E-05   -0.136497E-05   -0.154509E-05   -0.177336E-05   -0.205848E-05   -0.236173E-05   -0.270255E-05   -0.311796E-05   -0.359776E-05   -0.413599E-05   -0.473572E-05   -0.543174E-05   -0.623969E-05   -0.717534E-05   -0.822930E-05   -0.941968E-05   -0.107839E-04   -0.123461E-04   -0.141432E-04   -0.161837E-04   -0.184939E-04   -0.211233E-04   -0.241254E-04   -0.275404E-04   -0.314283E-04   -0.358546E-04   -0.408900E-04   -0.466055E-04   -0.530904E-04   -0.604504E-04   -0.688026E-04   -0.782732E-04   -0.889991E-04   -0.101140E-03   -0.114872E-03   -0.130384E-03   -0.147898E-03   -0.167652E-03   -0.189937E-03   -0.215043E-03   -0.243292E-03   -0.275057E-03   -0.310743E-03   -0.350800E-03   -0.395720E-03   -0.446040E-03   -0.502348E-03   -0.565292E-03   -0.635581E-03   -0.713984E-03   -0.801332E-03   -0.898526E-03   -0.100654E-02   -0.112643E-02   -0.125932E-02   -0.140642E-02   -0.156901E-02   -0.174846E-02   -0.194620E-02   -0.216376E-02   -0.240273E-02   -0.266478E-02   -0.295163E-02   -0.326505E-02   -0.360691E-02   -0.397903E-02   -0.438330E-02   -0.482162E-02   -0.529584E-02   -0.580780E-02   -0.635927E-02   -0.695195E-02   -0.758739E-02   -0.826703E-02   -0.899212E-02   -0.976369E-02   -0.105825E-01   -0.114492E-01   -0.123638E-01   -0.133262E-01   -0.143359E-01   -0.153919E-01   -0.164928E-01   -0.176366E-01   -0.188210E-01   -0.200431E-01   -0.212991E-01   -0.225851E-01   -0.238973E-01   -0.252311E-01   -0.265754E-01   -0.279270E-01   -0.292946E-01   -0.306514E-01   -0.319950E-01   -0.333195E-01   -0.346149E-01   -0.358759E-01   -0.370939E-01   -0.382607E-01   -0.393686E-01   -0.404102E-01   -0.413781E-01   -0.422642E-01   -0.430609E-01   -0.437627E-01   -0.443682E-01   -0.448782E-01   -0.452871E-01   -0.455650E-01   -0.456935E-01   -0.457252E-01   -0.456935E-01   -0.455650E-01   -0.452871E-01   -0.448782E-01   -0.443682E-01   -0.437627E-01   -0.430609E-01   -0.422642E-01   -0.413781E-01   -0.404102E-01   -0.393686E-01   -0.382607E-01   -0.370939E-01   -0.358759E-01   -0.346149E-01   -0.333195E-01   -0.319950E-01   -0.306514E-01   -0.292946E-01   -0.279270E-01   -0.265754E-01   -0.252311E-01   -0.238973E-01   -0.225851E-01   -0.212991E-01   -0.200431E-01   -0.188210E-01   -0.176366E-01   -0.164928E-01   -0.153919E-01   -0.143359E-01   -0.133262E-01   -0.123638E-01   -0.114492E-01   -0.105825E-01   -0.976369E-02   -0.899212E-02   -0.826704E-02   -0.758739E-02   -0.695195E-02   -0.635927E-02   -0.580780E-02   -0.529584E-02   -0.482162E-02   -0.438330E-02   -0.397903E-02   -0.360691E-02   -0.326505E-02   -0.295163E-02   -0.266478E-02   -0.240273E-02   -0.216376E-02   -0.194620E-02   -0.174846E-02   -0.156901E-02   -0.140642E-02   -0.125932E-02   -0.112643E-02   -0.100654E-02   -0.898526E-03   -0.801333E-03   -0.713985E-03   -0.635582E-03   -0.565292E-03   -0.502349E-03   -0.446041E-03   -0.395721E-03   -0.350801E-03   -0.310743E-03   -0.275057E-03   -0.243292E-03   -0.215043E-03   -0.189937E-03   -0.167652E-03   -0.147897E-03   -0.130383E-03   -0.114870E-03   -0.101139E-03   -0.889968E-04   -0.782709E-04   -0.688010E-04   -0.604482E-04   -0.530896E-04   -0.466076E-04   -0.408921E-04   -0.358537E-04   -0.314279E-04   -0.275443E-04   -0.241320E-04   -0.211298E-04   -0.184969E-04   -0.161763E-04   -0.141412E-04   -0.123599E-04   -0.107908E-04   -0.943098E-05   -0.823482E-05   -0.717534E-05   -0.625193E-05   -0.544503E-05   -0.473794E-05   -0.412172E-05   -0.356740E-05   -0.308589E-05   -0.271524E-05   -0.238797E-05   -0.207924E-05   -0.177447E-05   -0.153500E-05   -0.137127E-05   -0.121061E-05   -0.102729E-05   -0.863602E-06   -0.729414E-06   -0.636982E-06   -0.574655E-06   -0.467208E-06   -0.373953E-06   -0.310523E-06   -0.265863E-06   -0.283279E-06   -0.289816E-06   -0.235032E-06   -0.146098E-06   -0.818967E-07   -0.910248E-07   -0.127687E-06   -0.143742E-06   -0.181208E-06   -0.193866E-06   -0.126029E-06   -0.607151E-07   -0.676629E-08    0.459368E-07    0.330189E-08   -0.585058E-07   -0.568347E-07   -0.547540E-08    0.394257E-07    0.187043E-07    0.530845E-08    0.335159E-07    0.210619E-07   -0.592217E-07   -0.303054E-07    0.249156E-07    0.345130E-07    0.133214E-08   -0.800218E-08    0.197049E-07   -0.381084E-08    0.220936E-07   -0.406661E-07   -0.502859E-07    0.138074E-07    0.305977E-07    0.133714E-07    0.162652E-07    0.406131E-07    0.497889E-07    0.544327E-07    0.602489E-07    0.421851E-07   -0.655496E-08   -0.666280E-07   -0.685340E-07   -0.289730E-07   -0.209081E-07    0.125114E-07    0.684504E-07    0.972422E-07    0.329112E-07   -0.503201E-07   -0.914355E-07   -0.688374E-07   -0.187943E-07    0.608812E-08    0.384358E-07    0.793996E-07    0.831636E-07    0.951214E-08   -0.419420E-07   -0.880056E-07   -0.117966E-06   -0.887392E-07    0.608430E-08    0.865081E-07    0.627000E-07   -0.313353E-07   -0.965983E-07   -0.511543E-07    0.145939E-07    0.835815E-07    0.104196E-06    0.117584E-07   -0.363720E-07   -0.500153E-07   -0.469978E-07   -0.887187E-08    0.740349E-07    0.707971E-07   -0.159814E-07   -0.105235E-06   -0.825761E-07    0.385412E-08    0.604969E-07    0.559995E-07    0.136028E-07   -0.611353E-07   -0.107055E-06   -0.485498E-07   -0.870359E-09    0.594435E-07    0.201136E-07   -0.519615E-07   -0.929957E-07   -0.623953E-07   -0.290340E-08    0.572289E-07    0.837256E-07    0.848998E-07    0.256161E-07   -0.334347E-07   -0.743709E-07   -0.518353E-07   -0.213192E-07    0.577678E-08    0.628525E-07    0.161502E-07   -0.699367E-07   -0.927898E-07   -0.289846E-07   -0.122917E-08   -0.164549E-07   -0.138486E-07   -0.338664E-07   -0.459918E-07   -0.372331E-07    0.649753E-08    0.499435E-07    0.306088E-07   -0.158438E-08   -0.866671E-08    0.268501E-07    0.298320E-07   -0.282160E-07   -0.811647E-07   -0.722003E-07   -0.207232E-07
   -0.339909E-07    0.358749E-07    0.587911E-07    0.212184E-07   -0.859902E-07   -0.165358E-06   -0.141265E-06   -0.267459E-08    0.894392E-07    0.368367E-07   -0.422009E-07   -0.595034E-07    0.379870E-08    0.349192E-07    0.109388E-07   -0.190885E-07   -0.283507E-07   -0.149404E-07   -0.318034E-08   -0.248512E-07   -0.625180E-07   -0.336676E-07    0.458928E-07    0.718956E-07    0.423826E-07   -0.486977E-07   -0.706578E-07   -0.107132E-07    0.385368E-07    0.213267E-07   -0.216384E-07    0.152317E-07    0.883231E-07    0.131455E-06    0.816120E-07   -0.409054E-07   -0.189887E-06   -0.162097E-06   -0.168788E-07    0.118880E-06    0.161387E-06    0.578567E-07   -0.432996E-07   -0.547122E-07    0.316060E-07    0.833476E-07    0.300079E-07   -0.630857E-07   -0.587934E-07    0.301428E-07    0.979274E-07    0.248290E-07   -0.440631E-07   -0.419158E-07   -0.113431E-08    0.257176E-07    0.509196E-07    0.109239E-06    0.947763E-07   -0.360380E-07   -0.115876E-06   -0.106591E-06   -0.217004E-07    0.299943E-07    0.225230E-07    0.611427E-07    0.989252E-07    0.247859E-07   -0.596327E-07   -0.659551E-07   -0.163846E-07    0.847739E-07    0.171026E-06    0.122569E-06   -0.347017E-08   -0.114792E-06   -0.161132E-06   -0.128151E-06   -0.101605E-07    0.112389E-06    0.175895E-06    0.125477E-06   -0.713570E-08   -0.117323E-06   -0.103018E-06   -0.329518E-07    0.252204E-07    0.831128E-07    0.869814E-07    0.520853E-08   -0.319358E-07    0.634073E-08    0.510285E-07    0.118482E-06    0.119191E-06    0.281424E-07   -0.993467E-07   -0.186576E-06   -0.128964E-06    0.283498E-08    0.848808E-07    0.677615E-07    0.104231E-08   -0.476158E-07   -0.699391E-07   -0.503002E-07    0.168812E-07    0.656364E-07    0.673110E-07    0.474118E-07    0.194882E-07   -0.245830E-07   -0.112027E-06   -0.164049E-06   -0.121564E-06   -0.267442E-07    0.293148E-07    0.161310E-07   -0.582301E-07   -0.150092E-06   -0.226424E-06   -0.230406E-06   -0.218817E-06   -0.171373E-06   -0.125105E-06   -0.107288E-06   -0.180792E-06   -0.253464E-06   -0.309390E-06   -0.338909E-06   -0.446816E-06   -0.556542E-06   -0.648153E-06   -0.764717E-06   -0.896806E-06   -0.998506E-06   -0.113856E-05   -0.131985E-05   -0.154288E-05   -0.180540E-05   -0.210512E-05   -0.238561E-05   -0.271844E-05   -0.313153E-05   -0.360466E-05   -0.412967E-05   -0.475771E-05   -0.547623E-05   -0.631056E-05   -0.723876E-05   -0.830820E-05   -0.954187E-05   -0.109713E-04   -0.125911E-04   -0.144388E-04   -0.165477E-04   -0.189463E-04   -0.217018E-04   -0.248501E-04   -0.284179E-04   -0.324849E-04   -0.371420E-04   -0.424580E-04   -0.484982E-04   -0.553707E-04   -0.632054E-04   -0.721259E-04   -0.822479E-04   -0.937606E-04   -0.106854E-03   -0.121709E-03   -0.138562E-03   -0.157669E-03   -0.179310E-03   -0.203780E-03   -0.231440E-03   -0.262710E-03   -0.298032E-03   -0.337878E-03   -0.382787E-03   -0.433359E-03   -0.490260E-03   -0.554227E-03   -0.626077E-03   -0.706690E-03   -0.797040E-03   -0.898204E-03   -0.101136E-02   -0.113778E-02   -0.127886E-02   -0.143610E-02   -0.161114E-02   -0.180575E-02   -0.202184E-02   -0.226143E-02   -0.252670E-02   -0.281997E-02   -0.314370E-02   -0.350048E-02   -0.389305E-02   -0.432426E-02   -0.479708E-02   -0.531459E-02   -0.587998E-02   -0.649646E-02   -0.716731E-02   -0.789585E-02   -0.868535E-02   -0.953904E-02   -0.104601E-01   -0.114514E-01   -0.125159E-01   -0.136561E-01   -0.148743E-01   -0.161724E-01   -0.175519E-01   -0.190138E-01   -0.205586E-01   -0.221861E-01   -0.238955E-01   -0.256853E-01   -0.275531E-01   -0.294957E-01   -0.315091E-01   -0.335882E-01   -0.357270E-01   -0.379186E-01   -0.401562E-01   -0.424320E-01   -0.447273E-01   -0.470351E-01   -0.493730E-01   -0.516947E-01   -0.539929E-01   -0.562598E-01   -0.584777E-01   -0.606371E-01   -0.627233E-01   -0.647222E-01   -0.666207E-01   -0.684060E-01   -0.700650E-01   -0.715840E-01   -0.729497E-01   -0.741533E-01   -0.751932E-01   -0.760698E-01   -0.767708E-01   -0.772425E-01   -0.774601E-01   -0.775125E-01   -0.774601E-01   -0.772425E-01   -0.767708E-01   -0.760698E-01   -0.751932E-01   -0.741533E-01   -0.729497E-01   -0.715840E-01   -0.700650E-01   -0.684060E-01   -0.666207E-01   -0.647222E-01   -0.627233E-01   -0.606371E-01   -0.584777E-01   -0.562598E-01   -0.539930E-01   -0.516947E-01   -0.493730E-01   -0.470351E-01   -0.447273E-01   -0.424320E-01   -0.401562E-01   -0.379186E-01   -0.357270E-01   -0.335882E-01   -0.315091E-01   -0.294957E-01   -0.275531E-01   -0.256853E-01   -0.238955E-01   -0.221861E-01   -0.205586E-01   -0.190138E-01   -0.175519E-01   -0.161724E-01   -0.148743E-01   -0.136561E-01   -0.125159E-01   -0.114514E-01   -0.104601E-01   -0.953904E-02   -0.868535E-02   -0.789585E-02   -0.716731E-02   -0.649646E-02   -0.587998E-02   -0.531459E-02   -0.479708E-02   -0.432426E-02   -0.389305E-02   -0.350048E-02   -0.314370E-02   -0.281997E-02   -0.252670E-02   -0.226142E-02   -0.202184E-02   -0.180575E-02   -0.161114E-02   -0.143610E-02   -0.127886E-02   -0.113778E-02   -0.101136E-02   -0.898204E-03   -0.797040E-03   -0.706691E-03   -0.626077E-03   -0.554228E-03   -0.490260E-03   -0.433360E-03   -0.382787E-03   -0.337878E-03   -0.298032E-03   -0.262711E-03   -0.231441E-03   -0.203779E-03   -0.179308E-03   -0.157667E-03   -0.138559E-03   -0.121705E-03   -0.106850E-03   -0.937585E-04   -0.822451E-04   -0.721279E-04   -0.632123E-04   -0.553710E-04   -0.484950E-04   -0.424619E-04   -0.371536E-04   -0.324981E-04   -0.284239E-04   -0.248466E-04   -0.216966E-04   -0.189426E-04   -0.165427E-04   -0.144586E-04   -0.126136E-04   -0.109936E-04   -0.956846E-05   -0.831831E-05   -0.723069E-05   -0.628219E-05   -0.542438E-05   -0.472178E-05   -0.413040E-05   -0.362219E-05   -0.316737E-05   -0.272307E-05   -0.237596E-05   -0.211413E-05   -0.185553E-05   -0.157254E-05   -0.131835E-05   -0.112896E-05   -0.100605E-05   -0.907646E-06   -0.760371E-06   -0.609427E-06   -0.494414E-06   -0.425570E-06   -0.446915E-06   -0.459260E-06   -0.378341E-06   -0.235731E-06   -0.115410E-06   -0.752937E-07   -0.133587E-06   -0.192550E-06   -0.253800E-06   -0.310099E-06   -0.236302E-06   -0.135601E-06   -0.328200E-07    0.477023E-07   -0.107290E-09   -0.839724E-07   -0.839256E-07   -0.119941E-07    0.502888E-07    0.491705E-08   -0.280186E-07    0.569559E-09    0.254859E-08   -0.781070E-07   -0.794271E-07    0.192614E-07    0.673388E-07    0.126927E-07   -0.352115E-07   -0.341979E-08    0.671950E-08   -0.466496E-08   -0.556639E-07   -0.640718E-07   -0.143710E-07    0.213572E-07    0.237582E-07    0.545037E-07    0.840586E-07    0.769608E-07    0.736305E-07    0.376354E-07    0.402320E-07   -0.208712E-07   -0.100545E-06   -0.668832E-07   -0.280675E-07   -0.559087E-07   -0.161168E-07    0.858118E-07    0.133745E-06    0.505692E-07   -0.709063E-07   -0.175418E-06   -0.148704E-06   -0.559908E-07    0.264537E-07    0.741986E-07    0.114073E-06    0.124657E-06    0.588912E-07   -0.356444E-07   -0.137266E-06   -0.191595E-06   -0.119701E-06    0.199071E-07    0.111062E-06    0.738313E-07   -0.687209E-07   -0.163613E-06   -0.108647E-06    0.156674E-07    0.132949E-06    0.121772E-06    0.227118E-07   -0.661391E-07   -0.116140E-06   -0.101392E-06    0.174942E-07    0.118605E-06    0.109350E-06   -0.531355E-07   -0.152886E-06   -0.102738E-06    0.384355E-08    0.911110E-07    0.100468E-06    0.233121E-07   -0.100398E-06   -0.163877E-06   -0.940302E-07   -0.329399E-08    0.679556E-07    0.174672E-07   -0.864709E-07   -0.144815E-06   -0.959227E-07   -0.559383E-08    0.699319E-07    0.107582E-06    0.741349E-07    0.382070E-08   -0.795627E-07   -0.127771E-06   -0.770264E-07   -0.388753E-07    0.253358E-07    0.607022E-07    0.165984E-07   -0.825140E-07   -0.850162E-07   -0.120665E-08    0.302934E-07    0.932569E-08   -0.299949E-07   -0.760222E-07   -0.748601E-07   -0.477093E-07    0.341479E-07    0.850819E-07    0.188453E-07   -0.480806E-07   -0.132449E-07    0.664012E-07    0.735072E-07   -0.812449E-09   -0.705410E-07   -0.720208E-07   -0.560352E-07
   -0.448524E-07    0.892067E-07    0.132991E-06    0.846673E-07   -0.634472E-07   -0.185576E-06   -0.157938E-06   -0.319074E-07    0.668351E-07    0.415890E-07   -0.283464E-07   -0.587575E-08    0.610539E-07    0.740529E-07    0.390847E-07   -0.370164E-07   -0.562303E-07   -0.255232E-07    0.521875E-08   -0.438531E-07   -0.119632E-06   -0.690526E-07    0.649681E-07    0.152648E-06    0.131940E-06    0.112380E-07   -0.701276E-07   -0.624942E-07   -0.179961E-07   -0.258699E-07   -0.214468E-07    0.877910E-07    0.181932E-06    0.157989E-06    0.362510E-07   -0.123586E-06   -0.259260E-06   -0.191785E-06   -0.914927E-08    0.164781E-06    0.199999E-06    0.960668E-07   -0.306587E-07   -0.533508E-07    0.181194E-08    0.596020E-07   -0.239699E-08   -0.824471E-07   -0.157651E-07    0.121641E-06    0.168822E-06    0.759848E-07   -0.305833E-07   -0.364200E-07    0.153828E-07    0.621399E-07    0.809430E-07    0.117786E-06    0.842096E-07   -0.195553E-07   -0.845825E-07   -0.630396E-07   -0.212168E-07    0.124424E-07    0.233432E-07    0.102554E-06    0.161126E-06    0.811570E-07   -0.451108E-07   -0.832647E-07    0.149198E-07    0.149469E-06    0.193858E-06    0.108256E-06   -0.476180E-07   -0.186058E-06   -0.243128E-06   -0.152616E-06    0.459277E-07    0.202025E-06    0.233449E-06    0.147658E-06    0.718734E-08   -0.914952E-07   -0.877784E-07   -0.373965E-07    0.286929E-07    0.106381E-06    0.113435E-06    0.633922E-08   -0.520151E-07   -0.620190E-08    0.739782E-07    0.152289E-06    0.187620E-06    0.568101E-07   -0.117117E-06   -0.209477E-06   -0.147327E-06    0.157017E-07    0.116225E-06    0.116988E-06   -0.981042E-08   -0.126105E-06   -0.167834E-06   -0.117330E-06    0.211315E-07    0.127391E-06    0.137844E-06    0.106745E-06    0.466600E-07   -0.423064E-07   -0.116392E-06   -0.149625E-06   -0.942654E-07    0.141292E-07    0.580393E-07    0.397921E-07   -0.582074E-07   -0.205545E-06   -0.302641E-06   -0.311847E-06   -0.251643E-06   -0.138777E-06   -0.627926E-07   -0.954162E-07   -0.232070E-06   -0.339842E-06   -0.409905E-06   -0.459961E-06   -0.559579E-06   -0.694393E-06   -0.833186E-06   -0.100549E-05   -0.114943E-05   -0.127269E-05   -0.143345E-05   -0.164778E-05   -0.194135E-05   -0.228613E-05   -0.265151E-05   -0.301305E-05   -0.343180E-05   -0.395952E-05   -0.456607E-05   -0.525953E-05   -0.606213E-05   -0.699119E-05   -0.805794E-05   -0.924624E-05   -0.106275E-04   -0.122175E-04   -0.140533E-04   -0.161328E-04   -0.185331E-04   -0.212732E-04   -0.243850E-04   -0.279355E-04   -0.320108E-04   -0.366531E-04   -0.419517E-04   -0.480241E-04   -0.549640E-04   -0.628617E-04   -0.718458E-04   -0.820865E-04   -0.937567E-04   -0.107041E-03   -0.122163E-03   -0.139365E-03   -0.158904E-03   -0.181096E-03   -0.206284E-03   -0.234841E-03   -0.267173E-03   -0.303773E-03   -0.345204E-03   -0.392050E-03   -0.444962E-03   -0.504680E-03   -0.572010E-03   -0.647865E-03   -0.733253E-03   -0.829284E-03   -0.937173E-03   -0.105826E-02   -0.119402E-02   -0.134607E-02   -0.151618E-02   -0.170627E-02   -0.191844E-02   -0.215495E-02   -0.241826E-02   -0.271102E-02   -0.303608E-02   -0.339649E-02   -0.379550E-02   -0.423657E-02   -0.472336E-02   -0.525974E-02   -0.584975E-02   -0.649760E-02   -0.720769E-02   -0.798456E-02   -0.883281E-02   -0.975716E-02   -0.107624E-01   -0.118532E-01   -0.130342E-01   -0.143101E-01   -0.156851E-01   -0.171635E-01   -0.187490E-01   -0.204449E-01   -0.222542E-01   -0.241791E-01   -0.262213E-01   -0.283815E-01   -0.306597E-01   -0.330549E-01   -0.355650E-01   -0.381869E-01   -0.409161E-01   -0.437470E-01   -0.466726E-01   -0.496842E-01   -0.527724E-01   -0.559275E-01   -0.591378E-01   -0.623782E-01   -0.656374E-01   -0.689390E-01   -0.722223E-01   -0.754728E-01   -0.786802E-01   -0.818195E-01   -0.848761E-01   -0.878302E-01   -0.906612E-01   -0.933507E-01   -0.958803E-01   -0.982314E-01   -0.100384E+00   -0.102319E+00   -0.104026E+00   -0.105502E+00   -0.106747E+00   -0.107741E+00   -0.108404E+00   -0.108707E+00   -0.108778E+00   -0.108707E+00   -0.108404E+00   -0.107741E+00   -0.106747E+00   -0.105502E+00   -0.104026E+00   -0.102319E+00   -0.100384E+00   -0.982314E-01   -0.958803E-01   -0.933507E-01   -0.906612E-01   -0.878302E-01   -0.848761E-01   -0.818195E-01   -0.786802E-01   -0.754728E-01   -0.722223E-01   -0.689390E-01   -0.656374E-01   -0.623782E-01   -0.591378E-01   -0.559275E-01   -0.527724E-01   -0.496842E-01   -0.466726E-01   -0.437470E-01   -0.409161E-01   -0.381869E-01   -0.355650E-01   -0.330549E-01   -0.306597E-01   -0.283815E-01   -0.262213E-01   -0.241791E-01   -0.222542E-01   -0.204449E-01   -0.187490E-01   -0.171635E-01   -0.156851E-01   -0.143101E-01   -0.130342E-01   -0.118532E-01   -0.107624E-01   -0.975716E-02   -0.883281E-02   -0.798456E-02   -0.720769E-02   -0.649760E-02   -0.584975E-02   -0.525974E-02   -0.472336E-02   -0.423657E-02   -0.379550E-02   -0.339649E-02   -0.303608E-02   -0.271102E-02   -0.241825E-02   -0.215495E-02   -0.191844E-02   -0.170627E-02   -0.151618E-02   -0.134607E-02   -0.119402E-02   -0.105826E-02   -0.937173E-03   -0.829284E-03   -0.733253E-03   -0.647865E-03   -0.572010E-03   -0.504679E-03   -0.444962E-03   -0.392049E-03   -0.345205E-03   -0.303774E-03   -0.267172E-03   -0.234839E-03   -0.206282E-03   -0.181093E-03   -0.158902E-03   -0.139362E-03   -0.122160E-03   -0.107036E-03   -0.937553E-04   -0.820916E-04   -0.718468E-04   -0.628555E-04   -0.549674E-04   -0.480391E-04   -0.419677E-04   -0.366574E-04   -0.320109E-04   -0.279302E-04   -0.243606E-04   -0.212674E-04   -0.185616E-04   -0.161624E-04   -0.140797E-04   -0.122391E-04   -0.106349E-04   -0.924426E-05   -0.800621E-05   -0.692890E-05   -0.604758E-05   -0.525962E-05   -0.457708E-05   -0.398620E-05   -0.344440E-05   -0.300123E-05   -0.264952E-05   -0.232124E-05   -0.197889E-05   -0.165489E-05   -0.141218E-05   -0.126904E-05   -0.111871E-05   -0.958384E-06   -0.789570E-06   -0.612422E-06   -0.528033E-06   -0.534442E-06   -0.550771E-06   -0.464594E-06   -0.296793E-06   -0.141673E-06   -0.320343E-07   -0.855714E-07   -0.196869E-06   -0.287127E-06   -0.354247E-06   -0.287766E-06   -0.150092E-06   -0.242170E-07    0.592212E-07    0.130928E-07   -0.960467E-07   -0.110003E-06   -0.173484E-07    0.736489E-07    0.472810E-07    0.561912E-08    0.892464E-08    0.135360E-07   -0.678017E-07   -0.917859E-07    0.114424E-07    0.754305E-07    0.247134E-07   -0.454143E-07   -0.170291E-07    0.353826E-07    0.991317E-09   -0.766540E-08   -0.219464E-07   -0.359049E-07    0.457476E-08    0.590405E-07    0.106188E-06    0.134065E-06    0.914526E-07    0.677357E-07    0.400182E-07    0.612814E-07    0.608855E-08   -0.452417E-07    0.619793E-08    0.415655E-07   -0.149663E-07   -0.357967E-07    0.535808E-07    0.107446E-06    0.342734E-07   -0.869922E-07   -0.199519E-06   -0.154717E-06   -0.598978E-07    0.244414E-07    0.861956E-07    0.154800E-06    0.194990E-06    0.140868E-06    0.786732E-08   -0.116607E-06   -0.173612E-06   -0.944944E-07    0.597120E-07    0.131339E-06    0.696556E-07   -0.751755E-07   -0.162498E-06   -0.123350E-06    0.363154E-07    0.164201E-06    0.151886E-06    0.554309E-07   -0.652668E-07   -0.134828E-06   -0.858884E-07    0.794793E-07    0.180089E-06    0.132666E-06   -0.459076E-07   -0.152330E-06   -0.128557E-06   -0.205075E-07    0.111254E-06    0.166164E-06    0.987244E-07   -0.656203E-07   -0.162170E-06   -0.103157E-06    0.287618E-07    0.984976E-07    0.449217E-07   -0.750700E-07   -0.169929E-06   -0.100951E-06    0.184474E-07    0.111389E-06    0.142371E-06    0.713708E-07   -0.452088E-07   -0.136365E-06   -0.141081E-06   -0.504402E-07    0.297134E-07    0.955887E-07    0.622507E-07   -0.174395E-07   -0.699898E-07   -0.363931E-07    0.406289E-07    0.885139E-07    0.732306E-07   -0.699724E-08   -0.984331E-07   -0.119803E-06   -0.451660E-07    0.791174E-07    0.113327E-06    0.142335E-07   -0.578263E-07    0.148465E-07    0.132511E-06    0.149574E-06    0.678050E-07   -0.283161E-07   -0.399297E-07   -0.414983E-07
   -0.774479E-07    0.113111E-06    0.175444E-06    0.107871E-06   -0.490468E-07   -0.200723E-06   -0.222428E-06   -0.128809E-06   -0.148345E-07    0.226646E-07    0.544636E-08    0.557210E-07    0.106283E-06    0.854488E-07    0.475201E-08   -0.115704E-06   -0.133553E-06   -0.568701E-07   -0.899981E-08   -0.773569E-07   -0.156570E-06   -0.751329E-07    0.809259E-07    0.190219E-06    0.169389E-06    0.217632E-07   -0.114851E-06   -0.126668E-06   -0.710731E-07   -0.823219E-07   -0.443161E-07    0.116754E-06    0.195784E-06    0.117139E-06   -0.553367E-07   -0.218403E-06   -0.267760E-06   -0.162036E-06    0.131093E-07    0.177928E-06    0.222124E-06    0.145258E-06   -0.117558E-07   -0.910127E-07   -0.725313E-07   -0.208554E-07   -0.507046E-07   -0.793227E-07    0.306858E-07    0.168898E-06    0.195151E-06    0.741228E-07   -0.757504E-07   -0.603416E-07    0.959597E-08    0.363317E-07    0.353560E-07    0.645534E-07    0.527286E-07    0.460161E-08   -0.298347E-07   -0.117402E-07   -0.214180E-07   -0.454981E-07   -0.379559E-07    0.690499E-07    0.163995E-06    0.131915E-06    0.303093E-08   -0.595442E-07    0.451341E-07    0.155181E-06    0.128488E-06    0.570221E-08   -0.154201E-06   -0.271803E-06   -0.290704E-06   -0.135572E-06    0.901929E-07    0.249908E-06    0.215227E-06    0.893406E-07   -0.317490E-07   -0.793487E-07   -0.582443E-07   -0.142325E-07    0.353866E-07    0.942777E-07    0.112553E-06   -0.280882E-08   -0.105622E-06   -0.458664E-07    0.832573E-07    0.166024E-06    0.188246E-06    0.354387E-07   -0.140401E-06   -0.197368E-06   -0.133794E-06    0.403889E-07    0.120381E-06    0.900944E-07   -0.814288E-07   -0.231967E-06   -0.268660E-06   -0.181060E-06    0.107278E-08    0.157679E-06    0.214491E-06    0.164860E-06    0.418761E-07   -0.899337E-07   -0.166830E-06   -0.193328E-06   -0.119998E-06    0.178037E-07    0.793454E-07    0.502795E-07   -0.827628E-07   -0.278163E-06   -0.398673E-06   -0.392903E-06   -0.273277E-06   -0.137785E-06   -0.782586E-07   -0.145282E-06   -0.322071E-06   -0.458517E-06   -0.523871E-06   -0.607679E-06   -0.707259E-06   -0.869262E-06   -0.105182E-05   -0.125177E-05   -0.139415E-05   -0.153554E-05   -0.175337E-05   -0.201375E-05   -0.234988E-05   -0.274307E-05   -0.314817E-05   -0.359565E-05   -0.412374E-05   -0.475212E-05   -0.548919E-05   -0.634091E-05   -0.732785E-05   -0.846063E-05   -0.975212E-05   -0.111898E-04   -0.128513E-04   -0.147791E-04   -0.169976E-04   -0.195337E-04   -0.224477E-04   -0.257641E-04   -0.295646E-04   -0.339053E-04   -0.388720E-04   -0.445463E-04   -0.510351E-04   -0.584739E-04   -0.669809E-04   -0.766837E-04   -0.877194E-04   -0.100295E-03   -0.114644E-03   -0.131009E-03   -0.149650E-03   -0.170854E-03   -0.194957E-03   -0.222343E-03   -0.253453E-03   -0.288752E-03   -0.328779E-03   -0.374146E-03   -0.425533E-03   -0.483682E-03   -0.549431E-03   -0.623708E-03   -0.707546E-03   -0.802100E-03   -0.908648E-03   -0.102860E-02   -0.116351E-02   -0.131508E-02   -0.148521E-02   -0.167596E-02   -0.188960E-02   -0.212860E-02   -0.239564E-02   -0.269365E-02   -0.302582E-02   -0.339555E-02   -0.380655E-02   -0.426276E-02   -0.476843E-02   -0.532805E-02   -0.594641E-02   -0.662857E-02   -0.737983E-02   -0.820575E-02   -0.911209E-02   -0.101049E-01   -0.111902E-01   -0.123743E-01   -0.136634E-01   -0.150641E-01   -0.165824E-01   -0.182245E-01   -0.199963E-01   -0.219034E-01   -0.239510E-01   -0.261436E-01   -0.284853E-01   -0.309793E-01   -0.336280E-01   -0.364325E-01   -0.393932E-01   -0.425089E-01   -0.457770E-01   -0.491935E-01   -0.527530E-01   -0.564481E-01   -0.602696E-01   -0.642065E-01   -0.682464E-01   -0.723766E-01   -0.765812E-01   -0.808290E-01   -0.851043E-01   -0.894326E-01   -0.937441E-01   -0.980137E-01   -0.102229E+00   -0.106356E+00   -0.110376E+00   -0.114262E+00   -0.117988E+00   -0.121528E+00   -0.124859E+00   -0.127956E+00   -0.130791E+00   -0.133341E+00   -0.135590E+00   -0.137536E+00   -0.139179E+00   -0.140488E+00   -0.141358E+00   -0.141755E+00   -0.141847E+00   -0.141755E+00   -0.141358E+00   -0.140488E+00   -0.139179E+00   -0.137536E+00   -0.135590E+00   -0.133341E+00   -0.130791E+00   -0.127956E+00   -0.124859E+00   -0.121528E+00   -0.117988E+00   -0.114262E+00   -0.110376E+00   -0.106356E+00   -0.102229E+00   -0.980137E-01   -0.937441E-01   -0.894326E-01   -0.851043E-01   -0.808290E-01   -0.765812E-01   -0.723766E-01   -0.682464E-01   -0.642065E-01   -0.602696E-01   -0.564481E-01   -0.527530E-01   -0.491935E-01   -0.457770E-01   -0.425089E-01   -0.393932E-01   -0.364325E-01   -0.336280E-01   -0.309793E-01   -0.284853E-01   -0.261436E-01   -0.239510E-01   -0.219034E-01   -0.199963E-01   -0.182245E-01   -0.165824E-01   -0.150641E-01   -0.136634E-01   -0.123743E-01   -0.111902E-01   -0.101049E-01   -0.911209E-02   -0.820575E-02   -0.737983E-02   -0.662857E-02   -0.594641E-02   -0.532805E-02   -0.476843E-02   -0.426276E-02   -0.380655E-02   -0.339555E-02   -0.302582E-02   -0.269365E-02   -0.239564E-02   -0.212859E-02   -0.188960E-02   -0.167596E-02   -0.148521E-02   -0.131508E-02   -0.116351E-02   -0.102860E-02   -0.908648E-03   -0.802100E-03   -0.707546E-03   -0.623708E-03   -0.549430E-03   -0.483681E-03   -0.425532E-03   -0.374147E-03   -0.328779E-03   -0.288752E-03   -0.253452E-03   -0.222341E-03   -0.194956E-03   -0.170852E-03   -0.149646E-03   -0.131005E-03   -0.114642E-03   -0.100296E-03   -0.877195E-04   -0.766782E-04   -0.669815E-04   -0.584844E-04   -0.510535E-04   -0.445544E-04   -0.388689E-04   -0.338884E-04   -0.295443E-04   -0.257854E-04   -0.224728E-04   -0.195424E-04   -0.170018E-04   -0.147932E-04   -0.128656E-04   -0.111819E-04   -0.968961E-05   -0.841729E-05   -0.733158E-05   -0.636573E-05   -0.551542E-05   -0.476286E-05   -0.410593E-05   -0.356592E-05   -0.312697E-05   -0.275649E-05   -0.238959E-05   -0.200095E-05   -0.171012E-05   -0.151469E-05   -0.134328E-05   -0.116661E-05   -0.967827E-06   -0.778750E-06   -0.669524E-06   -0.636517E-06   -0.634100E-06   -0.544822E-06   -0.384858E-06   -0.193519E-06   -0.758103E-07   -0.120465E-06   -0.243123E-06   -0.332872E-06   -0.379946E-06   -0.316177E-06   -0.148966E-06   -0.292373E-08    0.520349E-07   -0.942643E-08   -0.154495E-06   -0.184841E-06   -0.858429E-07    0.422418E-07    0.853478E-07    0.741030E-07    0.546297E-07    0.242017E-08   -0.835027E-07   -0.115678E-06   -0.393944E-07    0.218228E-07   -0.625068E-08   -0.782769E-07   -0.586171E-07    0.143945E-07    0.207528E-07    0.558834E-07    0.153736E-07   -0.455798E-07   -0.337878E-07    0.403493E-07    0.928658E-07    0.950874E-07    0.205000E-07   -0.435951E-08    0.474480E-07    0.106392E-06    0.440794E-07    0.542991E-08    0.630559E-07    0.104673E-06    0.381799E-07   -0.330781E-07   -0.141953E-07    0.122215E-07   -0.472907E-07   -0.141160E-06   -0.201950E-06   -0.138332E-06   -0.755528E-07   -0.238332E-07    0.455549E-07    0.151252E-06    0.224682E-06    0.183482E-06    0.601262E-07   -0.578933E-07   -0.121077E-06   -0.748288E-07    0.304085E-07    0.788161E-07    0.283750E-07   -0.100360E-06   -0.151015E-06   -0.100357E-06    0.497559E-07    0.148880E-06    0.135493E-06    0.734343E-07   -0.408878E-07   -0.115642E-06   -0.695480E-07    0.850779E-07    0.173869E-06    0.102680E-06   -0.479647E-07   -0.177276E-06   -0.203943E-06   -0.974495E-07    0.756991E-07    0.184386E-06    0.142572E-06   -0.332452E-07   -0.138499E-06   -0.102467E-06    0.280283E-07    0.102307E-06    0.390282E-07   -0.877416E-07   -0.177428E-06   -0.101698E-06    0.417331E-07    0.136069E-06    0.132113E-06    0.267592E-07   -0.122445E-06   -0.209098E-06   -0.154641E-06   -0.126266E-07    0.121185E-06    0.153185E-06    0.460420E-07   -0.745163E-07   -0.104394E-06   -0.623425E-07    0.346181E-07    0.115842E-06    0.118302E-06    0.167708E-07   -0.120924E-06   -0.190276E-06   -0.112471E-06    0.364779E-07    0.827687E-07    0.325057E-08   -0.581581E-07    0.276681E-07    0.151428E-06    0.166680E-06    0.646376E-07   -0.383388E-07   -0.563523E-07   -0.412036E-07
   -0.872746E-07    0.107023E-06    0.219588E-06    0.168049E-06    0.137136E-07   -0.151332E-06   -0.240963E-06   -0.210482E-06   -0.103399E-06    0.361617E-08    0.688021E-07    0.136373E-06    0.178120E-06    0.112203E-06   -0.394854E-07   -0.181904E-06   -0.177338E-06   -0.648563E-07   -0.608298E-08   -0.507332E-07   -0.105387E-06   -0.115594E-07    0.142477E-06    0.224839E-06    0.180683E-06    0.318029E-07   -0.116845E-06   -0.141708E-06   -0.888254E-07   -0.845353E-07   -0.468265E-07    0.843605E-07    0.166414E-06    0.836593E-07   -0.126720E-06   -0.275521E-06   -0.223310E-06   -0.807969E-07    0.736861E-07    0.211513E-06    0.272239E-06    0.201780E-06    0.108669E-07   -0.116279E-06   -0.123160E-06   -0.833195E-07   -0.785148E-07   -0.444944E-07    0.782565E-07    0.187285E-06    0.179023E-06    0.382096E-07   -0.958402E-07   -0.619894E-07    0.118270E-07    0.528720E-08   -0.301991E-07   -0.150557E-07    0.126859E-07    0.403754E-07    0.431061E-07    0.691915E-07    0.261921E-07   -0.662541E-07   -0.105080E-06   -0.153184E-07    0.126601E-06    0.188046E-06    0.968799E-07    0.185794E-07    0.719028E-07    0.130206E-06    0.614574E-07   -0.835339E-07   -0.220338E-06   -0.290153E-06   -0.256845E-06   -0.687421E-07    0.130069E-06    0.242276E-06    0.165027E-06    0.291203E-07   -0.716608E-07   -0.693367E-07   -0.994651E-08    0.365761E-07    0.668739E-07    0.956587E-07    0.998196E-07   -0.656512E-08   -0.133030E-06   -0.679316E-07    0.950253E-07    0.181084E-06    0.156268E-06    0.183791E-08   -0.136917E-06   -0.163157E-06   -0.672982E-07    0.105933E-06    0.148858E-06    0.601448E-07   -0.145033E-06   -0.299983E-06   -0.296325E-06   -0.188318E-06    0.901417E-08    0.188380E-06    0.284133E-06    0.207995E-06    0.277750E-08   -0.157012E-06   -0.245700E-06   -0.250039E-06   -0.132873E-06    0.408419E-07    0.132433E-06    0.837840E-07   -0.105887E-06   -0.337893E-06   -0.453754E-06   -0.413956E-06   -0.277331E-06   -0.152092E-06   -0.136962E-06   -0.244957E-06   -0.417742E-06   -0.551820E-06   -0.618840E-06   -0.706364E-06   -0.832759E-06   -0.103317E-05   -0.125024E-05   -0.146655E-05   -0.164510E-05   -0.178630E-05   -0.202130E-05   -0.234372E-05   -0.273955E-05   -0.315758E-05   -0.362418E-05   -0.416130E-05   -0.479620E-05   -0.554225E-05   -0.639483E-05   -0.739725E-05   -0.855410E-05   -0.988025E-05   -0.113785E-04   -0.130777E-04   -0.150323E-04   -0.172893E-04   -0.198805E-04   -0.228584E-04   -0.262587E-04   -0.301448E-04   -0.346132E-04   -0.397312E-04   -0.455810E-04   -0.522641E-04   -0.599105E-04   -0.686757E-04   -0.787100E-04   -0.901693E-04   -0.103216E-03   -0.118090E-03   -0.135073E-03   -0.154452E-03   -0.176532E-03   -0.201657E-03   -0.230234E-03   -0.262718E-03   -0.299628E-03   -0.341547E-03   -0.389136E-03   -0.443107E-03   -0.504263E-03   -0.573516E-03   -0.651877E-03   -0.740465E-03   -0.840535E-03   -0.953483E-03   -0.108086E-02   -0.122436E-02   -0.138589E-02   -0.156751E-02   -0.177153E-02   -0.200046E-02   -0.225707E-02   -0.254437E-02   -0.286565E-02   -0.322450E-02   -0.362481E-02   -0.407077E-02   -0.456693E-02   -0.511815E-02   -0.572964E-02   -0.640699E-02   -0.715609E-02   -0.798321E-02   -0.889493E-02   -0.989815E-02   -0.110001E-01   -0.122081E-01   -0.135300E-01   -0.149734E-01   -0.165463E-01   -0.182568E-01   -0.201125E-01   -0.221213E-01   -0.242907E-01   -0.266277E-01   -0.291389E-01   -0.318303E-01   -0.347069E-01   -0.377731E-01   -0.410318E-01   -0.444850E-01   -0.481331E-01   -0.519749E-01   -0.560074E-01   -0.602259E-01   -0.646237E-01   -0.691920E-01   -0.739193E-01   -0.787922E-01   -0.837954E-01   -0.889128E-01   -0.941247E-01   -0.993940E-01   -0.104700E+00   -0.110068E+00   -0.115423E+00   -0.120727E+00   -0.125964E+00   -0.131095E+00   -0.136093E+00   -0.140926E+00   -0.145560E+00   -0.149964E+00   -0.154109E+00   -0.157963E+00   -0.161492E+00   -0.164666E+00   -0.167466E+00   -0.169890E+00   -0.171936E+00   -0.173565E+00   -0.174646E+00   -0.175138E+00   -0.175251E+00   -0.175138E+00   -0.174646E+00   -0.173565E+00   -0.171936E+00   -0.169890E+00   -0.167466E+00   -0.164666E+00   -0.161492E+00   -0.157963E+00   -0.154109E+00   -0.149964E+00   -0.145560E+00   -0.140926E+00   -0.136093E+00   -0.131095E+00   -0.125964E+00   -0.120727E+00   -0.115423E+00   -0.110068E+00   -0.104700E+00   -0.993940E-01   -0.941247E-01   -0.889128E-01   -0.837954E-01   -0.787922E-01   -0.739193E-01   -0.691920E-01   -0.646237E-01   -0.602259E-01   -0.560074E-01   -0.519749E-01   -0.481331E-01   -0.444850E-01   -0.410318E-01   -0.377731E-01   -0.347069E-01   -0.318303E-01   -0.291389E-01   -0.266277E-01   -0.242907E-01   -0.221213E-01   -0.201125E-01   -0.182568E-01   -0.165463E-01   -0.149734E-01   -0.135299E-01   -0.122081E-01   -0.110001E-01   -0.989815E-02   -0.889493E-02   -0.798321E-02   -0.715609E-02   -0.640699E-02   -0.572964E-02   -0.511815E-02   -0.456693E-02   -0.407077E-02   -0.362481E-02   -0.322450E-02   -0.286565E-02   -0.254437E-02   -0.225707E-02   -0.200046E-02   -0.177153E-02   -0.156751E-02   -0.138589E-02   -0.122436E-02   -0.108086E-02   -0.953483E-03   -0.840535E-03   -0.740465E-03   -0.651876E-03   -0.573516E-03   -0.504262E-03   -0.443106E-03   -0.389137E-03   -0.341548E-03   -0.299628E-03   -0.262717E-03   -0.230233E-03   -0.201656E-03   -0.176529E-03   -0.154451E-03   -0.135072E-03   -0.118090E-03   -0.103216E-03   -0.901652E-04   -0.787072E-04   -0.686815E-04   -0.599320E-04   -0.522853E-04   -0.455780E-04   -0.397073E-04   -0.346023E-04   -0.301692E-04   -0.262744E-04   -0.228518E-04   -0.198586E-04   -0.172846E-04   -0.150475E-04   -0.130762E-04   -0.113530E-04   -0.987483E-05   -0.856575E-05   -0.742579E-05   -0.641864E-05   -0.551049E-05   -0.472904E-05   -0.410668E-05   -0.359145E-05   -0.316271E-05   -0.274270E-05   -0.230681E-05   -0.197582E-05   -0.176295E-05   -0.158791E-05   -0.138808E-05   -0.114135E-05   -0.917982E-06   -0.783909E-06   -0.730655E-06   -0.704978E-06   -0.608497E-06   -0.461177E-06   -0.269829E-06   -0.160337E-06   -0.189271E-06   -0.288093E-06   -0.356422E-06   -0.374513E-06   -0.279513E-06   -0.102793E-06    0.451040E-07    0.815594E-07   -0.179806E-08   -0.174709E-06   -0.248970E-06   -0.173447E-06   -0.315702E-07    0.859914E-07    0.135178E-06    0.129080E-06    0.330770E-07   -0.687007E-07   -0.105427E-06   -0.771597E-07   -0.303401E-07   -0.341210E-07   -0.829856E-07   -0.772153E-07   -0.318925E-08    0.680980E-07    0.121107E-06    0.586957E-07   -0.267175E-07   -0.338431E-07    0.212497E-07    0.518273E-07    0.322761E-08   -0.690649E-07   -0.484904E-07    0.550619E-07    0.134014E-06    0.898842E-07    0.635322E-07    0.122462E-06    0.157869E-06    0.879940E-07   -0.745572E-08   -0.216634E-07   -0.232164E-07   -0.810794E-07   -0.159377E-06   -0.183881E-06   -0.120097E-06   -0.751870E-07   -0.466755E-07    0.660160E-08    0.109091E-06    0.208244E-06    0.204177E-06    0.121574E-06    0.223980E-07   -0.443034E-07   -0.441589E-07   -0.273578E-08    0.170709E-07   -0.311478E-07   -0.119801E-06   -0.120715E-06   -0.373793E-07    0.949256E-07    0.168439E-06    0.136717E-06    0.934272E-07   -0.102456E-08   -0.507737E-07   -0.185494E-07    0.731569E-07    0.131807E-06    0.772871E-07   -0.470601E-07   -0.193687E-06   -0.237732E-06   -0.138833E-06    0.528515E-07    0.181430E-06    0.153592E-06    0.349791E-09   -0.784551E-07   -0.535112E-07    0.322496E-07    0.779354E-07    0.755020E-08   -0.108428E-06   -0.126362E-06   -0.369734E-07    0.991018E-07    0.174555E-06    0.133353E-06   -0.594580E-08   -0.165521E-06   -0.240591E-06   -0.154842E-06    0.282916E-07    0.196006E-06    0.218873E-06    0.561982E-07   -0.103192E-06   -0.149661E-06   -0.831490E-07    0.567234E-07    0.175665E-06    0.188284E-06    0.713232E-07   -0.102187E-06   -0.213355E-06   -0.170324E-06   -0.352010E-07    0.448280E-07    0.195842E-07   -0.177251E-07    0.642388E-07    0.163437E-06    0.149981E-06    0.192675E-07   -0.725929E-07   -0.703182E-07   -0.579635E-08
   -0.917789E-07    0.586070E-07    0.204025E-06    0.221330E-06    0.991894E-07   -0.746918E-07   -0.230912E-06   -0.270681E-06   -0.186525E-06   -0.282900E-07    0.108142E-06    0.205731E-06    0.234191E-06    0.102601E-06   -0.929330E-07   -0.220767E-06   -0.189820E-06   -0.561871E-07    0.195796E-07   -0.670947E-08   -0.367938E-07    0.443242E-07    0.181392E-06    0.241120E-06    0.162975E-06    0.210363E-07   -0.101732E-06   -0.999621E-07   -0.655636E-07   -0.820332E-07   -0.621207E-07    0.146818E-07    0.105348E-06    0.472081E-07   -0.161903E-06   -0.294854E-06   -0.211664E-06   -0.315983E-07    0.135930E-06    0.248553E-06    0.305312E-06    0.216079E-06    0.861866E-08   -0.145535E-06   -0.176101E-06   -0.135877E-06   -0.928164E-07   -0.212323E-08    0.113529E-06    0.161366E-06    0.102042E-06   -0.188446E-07   -0.905048E-07   -0.603594E-07    0.113217E-07   -0.180883E-07   -0.107327E-06   -0.123605E-06   -0.401139E-07    0.244704E-07    0.737384E-07    0.129677E-06    0.796225E-07   -0.655773E-07   -0.174894E-06   -0.118172E-06    0.493724E-07    0.198722E-06    0.173386E-06    0.999756E-07    0.660939E-07    0.559137E-07   -0.145071E-07   -0.135186E-06   -0.234622E-06   -0.258147E-06   -0.175182E-06    0.475568E-08    0.148544E-06    0.166324E-06    0.770382E-07   -0.262618E-07   -0.959378E-07   -0.733110E-07    0.166143E-07    0.635024E-07    0.993218E-07    0.105924E-06    0.542324E-07   -0.344752E-07   -0.126304E-06   -0.762966E-07    0.774634E-07    0.151551E-06    0.914617E-07   -0.303195E-07   -0.139599E-06   -0.136253E-06   -0.120000E-07    0.129858E-06    0.169840E-06    0.427625E-07   -0.172673E-06   -0.319021E-06   -0.287604E-06   -0.155429E-06    0.348157E-07    0.197215E-06    0.276258E-06    0.164767E-06   -0.834099E-07   -0.247938E-06   -0.337586E-06   -0.299200E-06   -0.125440E-06    0.677030E-07    0.173032E-06    0.837089E-07   -0.147737E-06   -0.381699E-06   -0.477369E-06   -0.425477E-06   -0.309707E-06   -0.222944E-06   -0.233619E-06   -0.375495E-06   -0.546339E-06   -0.650151E-06   -0.705012E-06   -0.807620E-06   -0.938180E-06   -0.115033E-05   -0.142426E-05   -0.168378E-05   -0.188875E-05   -0.205033E-05   -0.227098E-05   -0.264344E-05   -0.311416E-05   -0.360275E-05   -0.414400E-05   -0.477076E-05   -0.550476E-05   -0.635826E-05   -0.733647E-05   -0.849568E-05   -0.981310E-05   -0.113026E-04   -0.130014E-04   -0.149608E-04   -0.172067E-04   -0.197994E-04   -0.227664E-04   -0.261571E-04   -0.300493E-04   -0.345304E-04   -0.396604E-04   -0.455304E-04   -0.522610E-04   -0.599635E-04   -0.687693E-04   -0.788450E-04   -0.903825E-04   -0.103583E-03   -0.118646E-03   -0.135825E-03   -0.155428E-03   -0.177805E-03   -0.203308E-03   -0.232342E-03   -0.265389E-03   -0.302975E-03   -0.345694E-03   -0.394242E-03   -0.449391E-03   -0.511957E-03   -0.582894E-03   -0.663265E-03   -0.754257E-03   -0.857184E-03   -0.973524E-03   -0.110492E-02   -0.125318E-02   -0.142033E-02   -0.160859E-02   -0.182040E-02   -0.205849E-02   -0.232582E-02   -0.262567E-02   -0.296161E-02   -0.333755E-02   -0.375774E-02   -0.422679E-02   -0.474971E-02   -0.533190E-02   -0.597915E-02   -0.669771E-02   -0.749421E-02   -0.837574E-02   -0.934979E-02   -0.104243E-01   -0.116074E-01   -0.129079E-01   -0.143348E-01   -0.158971E-01   -0.176044E-01   -0.194663E-01   -0.214923E-01   -0.236920E-01   -0.260748E-01   -0.286498E-01   -0.314256E-01   -0.344104E-01   -0.376114E-01   -0.410350E-01   -0.446864E-01   -0.485695E-01   -0.526868E-01   -0.570389E-01   -0.616247E-01   -0.664409E-01   -0.714820E-01   -0.767400E-01   -0.822045E-01   -0.878620E-01   -0.936965E-01   -0.996897E-01   -0.105822E+00   -0.112069E+00   -0.118390E+00   -0.124756E+00   -0.131192E+00   -0.137620E+00   -0.143989E+00   -0.150278E+00   -0.156441E+00   -0.162444E+00   -0.168250E+00   -0.173818E+00   -0.179111E+00   -0.184092E+00   -0.188724E+00   -0.192966E+00   -0.196781E+00   -0.200147E+00   -0.203062E+00   -0.205523E+00   -0.207481E+00   -0.208778E+00   -0.209367E+00   -0.209503E+00   -0.209367E+00   -0.208778E+00   -0.207481E+00   -0.205523E+00   -0.203062E+00   -0.200147E+00   -0.196781E+00   -0.192966E+00   -0.188724E+00   -0.184092E+00   -0.179111E+00   -0.173818E+00   -0.168250E+00   -0.162444E+00   -0.156441E+00   -0.150278E+00   -0.143989E+00   -0.137620E+00   -0.131192E+00   -0.124756E+00   -0.118390E+00   -0.112069E+00   -0.105822E+00   -0.996897E-01   -0.936965E-01   -0.878620E-01   -0.822045E-01   -0.767400E-01   -0.714820E-01   -0.664409E-01   -0.616247E-01   -0.570389E-01   -0.526868E-01   -0.485695E-01   -0.446864E-01   -0.410350E-01   -0.376114E-01   -0.344104E-01   -0.314256E-01   -0.286498E-01   -0.260748E-01   -0.236920E-01   -0.214923E-01   -0.194663E-01   -0.176044E-01   -0.158971E-01   -0.143348E-01   -0.129079E-01   -0.116074E-01   -0.104243E-01   -0.934979E-02   -0.837574E-02   -0.749421E-02   -0.669771E-02   -0.597915E-02   -0.533190E-02   -0.474971E-02   -0.422679E-02   -0.375774E-02   -0.333755E-02   -0.296161E-02   -0.262567E-02   -0.232582E-02   -0.205849E-02   -0.182040E-02   -0.160859E-02   -0.142033E-02   -0.125318E-02   -0.110492E-02   -0.973524E-03   -0.857184E-03   -0.754256E-03   -0.663264E-03   -0.582892E-03   -0.511957E-03   -0.449391E-03   -0.394244E-03   -0.345695E-03   -0.302976E-03   -0.265390E-03   -0.232344E-03   -0.203308E-03   -0.177806E-03   -0.155428E-03   -0.135823E-03   -0.118647E-03   -0.103581E-03   -0.903823E-04   -0.788526E-04   -0.687873E-04   -0.599864E-04   -0.522651E-04   -0.455222E-04   -0.396502E-04   -0.345239E-04   -0.300534E-04   -0.261561E-04   -0.227342E-04   -0.197726E-04   -0.172149E-04   -0.149758E-04   -0.130042E-04   -0.113072E-04   -0.980463E-05   -0.848132E-05   -0.731388E-05   -0.628094E-05   -0.541473E-05   -0.473030E-05   -0.413255E-05   -0.359276E-05   -0.307171E-05   -0.260501E-05   -0.224957E-05   -0.202688E-05   -0.184657E-05   -0.162334E-05   -0.133346E-05   -0.106411E-05   -0.900091E-06   -0.817284E-06   -0.762164E-06   -0.685611E-06   -0.562577E-06   -0.403706E-06   -0.283239E-06   -0.295110E-06   -0.346898E-06   -0.386329E-06   -0.373221E-06   -0.248597E-06   -0.853010E-07    0.629817E-07    0.117608E-06    0.176098E-07   -0.168542E-06   -0.289282E-06   -0.261301E-06   -0.142918E-06    0.156931E-07    0.134520E-06    0.168350E-06    0.848470E-07   -0.211139E-07   -0.699092E-07   -0.110888E-06   -0.917575E-07   -0.871087E-07   -0.110033E-06   -0.901750E-07   -0.162676E-07    0.892350E-07    0.129471E-06    0.776205E-07   -0.143015E-07   -0.256625E-07    0.698689E-08   -0.403321E-08   -0.101422E-06   -0.157724E-06   -0.845991E-07    0.293835E-07    0.103726E-06    0.979164E-07    0.894968E-07    0.140230E-06    0.153775E-06    0.818419E-07    0.418236E-08    0.119560E-07    0.115955E-07   -0.615535E-07   -0.154022E-06   -0.175517E-06   -0.118509E-06   -0.805051E-07   -0.627864E-07   -0.591419E-07    0.130066E-07    0.121875E-06    0.161928E-06    0.139268E-06    0.630044E-07   -0.130902E-07   -0.143457E-08    0.947890E-08   -0.177908E-07   -0.102280E-06   -0.168631E-06   -0.139967E-06   -0.239644E-07    0.103516E-06    0.174161E-06    0.160406E-06    0.949428E-07    0.123172E-07   -0.239634E-07    0.452559E-08    0.488738E-07    0.755178E-07    0.382913E-07   -0.608043E-07   -0.190042E-06   -0.236702E-06   -0.140411E-06    0.371410E-07    0.146140E-06    0.107325E-06    0.854537E-08   -0.335856E-07    0.112575E-07    0.515350E-07    0.173695E-07   -0.798746E-07   -0.158585E-06   -0.767280E-07    0.555943E-07    0.163790E-06    0.190971E-06    0.112170E-06   -0.441525E-07   -0.202504E-06   -0.271591E-06   -0.181035E-06    0.257196E-07    0.197408E-06    0.231468E-06    0.772725E-07   -0.956268E-07   -0.187342E-06   -0.855563E-07    0.957028E-07    0.227726E-06    0.242986E-06    0.950532E-07   -0.973568E-07   -0.210308E-06   -0.195046E-06   -0.888124E-07    0.112079E-07    0.311275E-07    0.212418E-07    0.994368E-07    0.164758E-06    0.971470E-07   -0.583725E-07   -0.149199E-06   -0.119858E-06    0.122704E-07
   -0.794728E-07    0.121276E-07    0.143228E-06    0.236370E-06    0.169769E-06    0.775815E-09   -0.181016E-06   -0.247631E-06   -0.203893E-06   -0.345647E-07    0.125833E-06    0.235906E-06    0.255357E-06    0.880939E-07   -0.134365E-06   -0.229286E-06   -0.161083E-06   -0.110301E-07    0.777684E-07    0.324937E-07    0.157183E-07    0.682476E-07    0.178910E-06    0.238443E-06    0.149088E-06    0.431677E-08   -0.850818E-07   -0.523113E-07   -0.161447E-07   -0.545700E-07   -0.808693E-07   -0.273783E-07    0.493744E-07    0.808259E-08   -0.156708E-06   -0.276746E-06   -0.210190E-06   -0.200962E-07    0.172075E-06    0.280297E-06    0.313450E-06    0.199220E-06   -0.129116E-08   -0.163678E-06   -0.221900E-06   -0.182804E-06   -0.837357E-07    0.528229E-07    0.142786E-06    0.129579E-06    0.270267E-07   -0.658167E-07   -0.654870E-07   -0.310959E-07    0.303584E-07   -0.158491E-07   -0.144308E-06   -0.182129E-06   -0.860663E-07   -0.193485E-07    0.749387E-07    0.155274E-06    0.103084E-06   -0.622508E-07   -0.198575E-06   -0.171960E-06   -0.212527E-07    0.175060E-06    0.225787E-06    0.164896E-06    0.494567E-07   -0.388580E-07   -0.883150E-07   -0.129535E-06   -0.172782E-06   -0.189419E-06   -0.747324E-07    0.764788E-07    0.143158E-06    0.790457E-07   -0.180537E-07   -0.628755E-07   -0.789748E-07   -0.343925E-07    0.475970E-07    0.828767E-07    0.109316E-06    0.101660E-06    0.259680E-07   -0.478909E-07   -0.992933E-07   -0.576624E-07    0.554197E-07    0.104393E-06    0.347484E-07   -0.498706E-07   -0.118337E-06   -0.926354E-07    0.117177E-07    0.111277E-06    0.147073E-06    0.428719E-07   -0.165434E-06   -0.284611E-06   -0.238610E-06   -0.971688E-07    0.735151E-07    0.201859E-06    0.217074E-06    0.653324E-07   -0.169930E-06   -0.319785E-06   -0.377189E-06   -0.311354E-06   -0.101277E-06    0.994758E-07    0.171800E-06    0.562103E-07   -0.170976E-06   -0.372769E-06   -0.466185E-06   -0.430105E-06   -0.364055E-06   -0.328440E-06   -0.358195E-06   -0.494474E-06   -0.638644E-06   -0.725751E-06   -0.784912E-06   -0.899169E-06   -0.104001E-05   -0.123677E-05   -0.154226E-05   -0.185215E-05   -0.208870E-05   -0.227752E-05   -0.252300E-05   -0.293636E-05   -0.346757E-05   -0.403792E-05   -0.466414E-05   -0.538064E-05   -0.620113E-05   -0.715356E-05   -0.826438E-05   -0.956386E-05   -0.110327E-04   -0.126953E-04   -0.145978E-04   -0.168001E-04   -0.193323E-04   -0.222471E-04   -0.255808E-04   -0.293970E-04   -0.337847E-04   -0.388288E-04   -0.446115E-04   -0.512341E-04   -0.588299E-04   -0.675343E-04   -0.774783E-04   -0.888510E-04   -0.101870E-03   -0.116781E-03   -0.133831E-03   -0.153287E-03   -0.175479E-03   -0.200808E-03   -0.229694E-03   -0.262601E-03   -0.300072E-03   -0.342709E-03   -0.391200E-03   -0.446331E-03   -0.508966E-03   -0.580067E-03   -0.660722E-03   -0.752145E-03   -0.855699E-03   -0.972899E-03   -0.110544E-02   -0.125521E-02   -0.142431E-02   -0.161504E-02   -0.182997E-02   -0.207194E-02   -0.234407E-02   -0.264982E-02   -0.299294E-02   -0.337760E-02   -0.380831E-02   -0.429001E-02   -0.482806E-02   -0.542828E-02   -0.609693E-02   -0.684079E-02   -0.766710E-02   -0.858364E-02   -0.959865E-02   -0.107209E-01   -0.119597E-01   -0.133246E-01   -0.148258E-01   -0.164739E-01   -0.182797E-01   -0.202542E-01   -0.224088E-01   -0.247548E-01   -0.273035E-01   -0.300660E-01   -0.330530E-01   -0.362749E-01   -0.397412E-01   -0.434606E-01   -0.474409E-01   -0.516882E-01   -0.562073E-01   -0.610013E-01   -0.660711E-01   -0.714155E-01   -0.770308E-01   -0.829108E-01   -0.890463E-01   -0.954249E-01   -0.102031E+00   -0.108847E+00   -0.115849E+00   -0.123016E+00   -0.130320E+00   -0.137712E+00   -0.145159E+00   -0.152683E+00   -0.160206E+00   -0.167659E+00   -0.175019E+00   -0.182234E+00   -0.189260E+00   -0.196056E+00   -0.202574E+00   -0.208771E+00   -0.214603E+00   -0.220026E+00   -0.224993E+00   -0.229460E+00   -0.233401E+00   -0.236816E+00   -0.239698E+00   -0.241991E+00   -0.243508E+00   -0.244194E+00   -0.244352E+00   -0.244194E+00   -0.243508E+00   -0.241991E+00   -0.239698E+00   -0.236816E+00   -0.233401E+00   -0.229460E+00   -0.224993E+00   -0.220026E+00   -0.214603E+00   -0.208771E+00   -0.202574E+00   -0.196056E+00   -0.189260E+00   -0.182234E+00   -0.175019E+00   -0.167659E+00   -0.160206E+00   -0.152683E+00   -0.145159E+00   -0.137712E+00   -0.130320E+00   -0.123016E+00   -0.115849E+00   -0.108847E+00   -0.102031E+00   -0.954249E-01   -0.890463E-01   -0.829108E-01   -0.770308E-01   -0.714155E-01   -0.660711E-01   -0.610013E-01   -0.562073E-01   -0.516882E-01   -0.474409E-01   -0.434606E-01   -0.397412E-01   -0.362749E-01   -0.330530E-01   -0.300660E-01   -0.273035E-01   -0.247548E-01   -0.224088E-01   -0.202542E-01   -0.182797E-01   -0.164739E-01   -0.148258E-01   -0.133246E-01   -0.119597E-01   -0.107209E-01   -0.959865E-02   -0.858364E-02   -0.766710E-02   -0.684079E-02   -0.609693E-02   -0.542828E-02   -0.482806E-02   -0.429001E-02   -0.380831E-02   -0.337760E-02   -0.299294E-02   -0.264982E-02   -0.234408E-02   -0.207194E-02   -0.182997E-02   -0.161504E-02   -0.142431E-02   -0.125521E-02   -0.110544E-02   -0.972898E-03   -0.855699E-03   -0.752145E-03   -0.660721E-03   -0.580067E-03   -0.508966E-03   -0.446332E-03   -0.391201E-03   -0.342710E-03   -0.300073E-03   -0.262603E-03   -0.229697E-03   -0.200812E-03   -0.175480E-03   -0.153283E-03   -0.133827E-03   -0.116784E-03   -0.101874E-03   -0.888582E-04   -0.774935E-04   -0.675492E-04   -0.588393E-04   -0.512377E-04   -0.446075E-04   -0.388086E-04   -0.337630E-04   -0.293801E-04   -0.255549E-04   -0.222284E-04   -0.193371E-04   -0.168135E-04   -0.146112E-04   -0.126846E-04   -0.109976E-04   -0.949259E-05   -0.818012E-05   -0.704623E-05   -0.612143E-05   -0.536712E-05   -0.469353E-05   -0.404195E-05   -0.342392E-05   -0.291965E-05   -0.254142E-05   -0.228374E-05   -0.207018E-05   -0.181121E-05   -0.149180E-05   -0.122728E-05   -0.102720E-05   -0.900697E-06   -0.820636E-06   -0.752939E-06   -0.655614E-06   -0.520706E-06   -0.407099E-06   -0.408139E-06   -0.420644E-06   -0.431357E-06   -0.376687E-06   -0.264185E-06   -0.108019E-06    0.588919E-07    0.142279E-06    0.483807E-07   -0.139635E-06   -0.293252E-06   -0.320233E-06   -0.235666E-06   -0.704306E-07    0.110420E-06    0.188925E-06    0.145399E-06    0.617720E-07   -0.298699E-07   -0.108861E-06   -0.128436E-06   -0.159651E-06   -0.182858E-06   -0.113409E-06   -0.100409E-07    0.110953E-06    0.147029E-06    0.881664E-07   -0.144722E-08   -0.180427E-07   -0.850643E-08   -0.484790E-07   -0.166698E-06   -0.214573E-06   -0.107245E-06    0.243588E-07    0.863794E-07    0.908746E-07    0.798565E-07    0.112180E-06    0.105388E-06    0.469153E-07    0.107376E-07    0.512362E-07    0.665251E-07   -0.134551E-07   -0.126800E-06   -0.152863E-06   -0.865840E-07   -0.474820E-07   -0.767834E-07   -0.137551E-06   -0.939729E-07   -0.704167E-08    0.732585E-07    0.101037E-06    0.509790E-07    0.154460E-07    0.617935E-07    0.794275E-07    0.171454E-07   -0.129148E-06   -0.206619E-06   -0.177388E-06   -0.580008E-07    0.661818E-07    0.155642E-06    0.185916E-06    0.108899E-06    0.213328E-07   -0.256170E-07   -0.252659E-07    0.126560E-07    0.378557E-07    0.830864E-08   -0.531448E-07   -0.147735E-06   -0.192384E-06   -0.101600E-06    0.201371E-07    0.907289E-07    0.521685E-07    0.206279E-07    0.259952E-07    0.922893E-07    0.868482E-07   -0.351884E-07   -0.169841E-06   -0.194979E-06   -0.403960E-07    0.122624E-06    0.206510E-06    0.189520E-06    0.679962E-07   -0.842026E-07   -0.212114E-06   -0.275701E-06   -0.199662E-06   -0.124068E-07    0.134491E-06    0.194692E-06    0.110098E-06   -0.329537E-07   -0.150952E-06   -0.683950E-07    0.113868E-06    0.244910E-06    0.251895E-06    0.934737E-07   -0.961735E-07   -0.197829E-06   -0.193394E-06   -0.815575E-07    0.278873E-07    0.372461E-07    0.447789E-07    0.118672E-06    0.153413E-06    0.552605E-07   -0.115245E-06   -0.208259E-06   -0.155086E-06    0.113346E-08
   -0.700790E-07   -0.139915E-07    0.668191E-07    0.181559E-06    0.159613E-06    0.361981E-07   -0.102380E-06   -0.157392E-06   -0.178249E-06   -0.637635E-07    0.103993E-06    0.205929E-06    0.212469E-06    0.540293E-07   -0.161700E-06   -0.229934E-06   -0.129057E-06    0.325171E-07    0.119151E-06    0.496608E-07    0.130758E-07    0.439834E-07    0.145965E-06    0.198890E-06    0.103639E-06   -0.363625E-07   -0.874458E-07   -0.358450E-07    0.655754E-08   -0.386392E-07   -0.859262E-07   -0.451862E-07    0.930258E-08   -0.553231E-07   -0.166661E-06   -0.251059E-06   -0.212563E-06   -0.468956E-07    0.140678E-06    0.256257E-06    0.275161E-06    0.157520E-06   -0.278837E-07   -0.190676E-06   -0.253302E-06   -0.209013E-06   -0.721025E-07    0.858408E-07    0.145227E-06    0.760296E-07   -0.274383E-07   -0.895921E-07   -0.341699E-07    0.155126E-07    0.412086E-07   -0.182187E-07   -0.143610E-06   -0.188868E-06   -0.130263E-06   -0.479831E-07    0.377317E-07    0.984138E-07    0.508785E-07   -0.901835E-07   -0.187985E-06   -0.162478E-06   -0.455599E-07    0.126246E-06    0.220933E-06    0.161902E-06    0.100648E-07   -0.127627E-06   -0.160659E-06   -0.110200E-06   -0.823949E-07   -0.100455E-06   -0.109615E-07    0.859325E-07    0.914280E-07   -0.840426E-08   -0.845954E-07   -0.724422E-07   -0.404796E-07    0.200588E-07    0.651720E-07    0.762134E-07    0.597372E-07    0.404995E-07   -0.796598E-08   -0.645409E-07   -0.858280E-07   -0.316908E-07    0.440616E-07    0.289155E-07   -0.322621E-07   -0.608269E-07   -0.854200E-07   -0.798503E-07   -0.320442E-07    0.384704E-07    0.762112E-07    0.102235E-07   -0.157721E-06   -0.251627E-06   -0.194759E-06   -0.469585E-07    0.101051E-06    0.195673E-06    0.148812E-06   -0.329297E-07   -0.257778E-06   -0.386051E-06   -0.391009E-06   -0.289056E-06   -0.865570E-07    0.750362E-07    0.100320E-06   -0.150521E-07   -0.190620E-06   -0.353956E-06   -0.459423E-06   -0.447103E-06   -0.422300E-06   -0.437022E-06   -0.484281E-06   -0.588681E-06   -0.708215E-06   -0.800060E-06   -0.887467E-06   -0.101814E-05   -0.117300E-05   -0.137612E-05   -0.166425E-05   -0.196136E-05   -0.222113E-05   -0.247750E-05   -0.279444E-05   -0.326183E-05   -0.383911E-05   -0.446828E-05   -0.517232E-05   -0.597061E-05   -0.686757E-05   -0.792503E-05   -0.915749E-05   -0.105784E-04   -0.121952E-04   -0.140464E-04   -0.161610E-04   -0.185975E-04   -0.214006E-04   -0.246325E-04   -0.283376E-04   -0.325785E-04   -0.374405E-04   -0.430312E-04   -0.494527E-04   -0.568211E-04   -0.652654E-04   -0.749336E-04   -0.859970E-04   -0.986546E-04   -0.113144E-03   -0.129740E-03   -0.148722E-03   -0.170408E-03   -0.195151E-03   -0.223388E-03   -0.255606E-03   -0.292333E-03   -0.334169E-03   -0.381791E-03   -0.435993E-03   -0.497635E-03   -0.567682E-03   -0.647239E-03   -0.737526E-03   -0.839915E-03   -0.955942E-03   -0.108733E-02   -0.123598E-02   -0.140405E-02   -0.159389E-02   -0.180812E-02   -0.204967E-02   -0.232174E-02   -0.262789E-02   -0.297203E-02   -0.335847E-02   -0.379192E-02   -0.427755E-02   -0.482096E-02   -0.542830E-02   -0.610619E-02   -0.686183E-02   -0.770294E-02   -0.863784E-02   -0.967542E-02   -0.108252E-01   -0.120972E-01   -0.135020E-01   -0.150508E-01   -0.167554E-01   -0.186277E-01   -0.206804E-01   -0.229261E-01   -0.253781E-01   -0.280493E-01   -0.309530E-01   -0.341018E-01   -0.375085E-01   -0.411849E-01   -0.451422E-01   -0.493905E-01   -0.539389E-01   -0.587947E-01   -0.639636E-01   -0.694491E-01   -0.752527E-01   -0.813728E-01   -0.878056E-01   -0.945438E-01   -0.101577E+00   -0.108891E+00   -0.116468E+00   -0.124287E+00   -0.132322E+00   -0.140548E+00   -0.148931E+00   -0.157419E+00   -0.165972E+00   -0.174606E+00   -0.183247E+00   -0.191808E+00   -0.200263E+00   -0.208550E+00   -0.216621E+00   -0.224429E+00   -0.231917E+00   -0.239036E+00   -0.245736E+00   -0.251966E+00   -0.257671E+00   -0.262802E+00   -0.267331E+00   -0.271254E+00   -0.274566E+00   -0.277200E+00   -0.278940E+00   -0.279726E+00   -0.279906E+00   -0.279726E+00   -0.278940E+00   -0.277200E+00   -0.274566E+00   -0.271254E+00   -0.267331E+00   -0.262802E+00   -0.257671E+00   -0.251966E+00   -0.245736E+00   -0.239036E+00   -0.231917E+00   -0.224429E+00   -0.216621E+00   -0.208550E+00   -0.200263E+00   -0.191808E+00   -0.183247E+00   -0.174606E+00   -0.165972E+00   -0.157419E+00   -0.148931E+00   -0.140548E+00   -0.132322E+00   -0.124287E+00   -0.116468E+00   -0.108891E+00   -0.101577E+00   -0.945438E-01   -0.878056E-01   -0.813728E-01   -0.752527E-01   -0.694491E-01   -0.639636E-01   -0.587947E-01   -0.539389E-01   -0.493905E-01   -0.451422E-01   -0.411849E-01   -0.375085E-01   -0.341018E-01   -0.309530E-01   -0.280493E-01   -0.253781E-01   -0.229261E-01   -0.206804E-01   -0.186277E-01   -0.167554E-01   -0.150508E-01   -0.135020E-01   -0.120972E-01   -0.108252E-01   -0.967542E-02   -0.863784E-02   -0.770294E-02   -0.686183E-02   -0.610619E-02   -0.542830E-02   -0.482096E-02   -0.427755E-02   -0.379192E-02   -0.335847E-02   -0.297203E-02   -0.262789E-02   -0.232174E-02   -0.204967E-02   -0.180812E-02   -0.159389E-02   -0.140405E-02   -0.123598E-02   -0.108733E-02   -0.955942E-03   -0.839915E-03   -0.737526E-03   -0.647239E-03   -0.567682E-03   -0.497635E-03   -0.435993E-03   -0.381792E-03   -0.334171E-03   -0.292337E-03   -0.255610E-03   -0.223392E-03   -0.195151E-03   -0.170403E-03   -0.148716E-03   -0.129740E-03   -0.113148E-03   -0.986547E-04   -0.860019E-04   -0.749451E-04   -0.652787E-04   -0.568350E-04   -0.494574E-04   -0.430131E-04   -0.374022E-04   -0.325340E-04   -0.283150E-04   -0.246179E-04   -0.213931E-04   -0.185951E-04   -0.161713E-04   -0.140321E-04   -0.121533E-04   -0.104797E-04   -0.904334E-05   -0.781836E-05   -0.682838E-05   -0.598982E-05   -0.524063E-05   -0.450978E-05   -0.383643E-05   -0.328506E-05   -0.286205E-05   -0.253187E-05   -0.225367E-05   -0.197206E-05   -0.165623E-05   -0.138482E-05   -0.114997E-05   -0.100613E-05   -0.917929E-06   -0.838982E-06   -0.733348E-06   -0.605981E-06   -0.508698E-06   -0.495539E-06   -0.520636E-06   -0.516545E-06   -0.427010E-06   -0.323758E-06   -0.168948E-06   -0.201387E-07    0.950739E-07    0.639468E-07   -0.109454E-06   -0.302302E-06   -0.377335E-06   -0.307544E-06   -0.131344E-06    0.882937E-07    0.201471E-06    0.180424E-06    0.999003E-07   -0.310496E-08   -0.809513E-07   -0.155579E-06   -0.245021E-06   -0.277451E-06   -0.186742E-06   -0.262111E-07    0.121366E-06    0.158482E-06    0.111744E-06    0.177307E-07   -0.350893E-07   -0.501082E-07   -0.798734E-07   -0.182045E-06   -0.231279E-06   -0.128115E-06    0.185555E-07    0.779595E-07    0.655243E-07    0.285885E-07    0.417770E-07    0.351680E-07   -0.121301E-07   -0.310667E-07    0.350870E-07    0.762274E-07    0.821252E-08   -0.101799E-06   -0.121310E-06   -0.505222E-07   -0.811298E-08   -0.794643E-07   -0.188799E-06   -0.203767E-06   -0.151333E-06   -0.515614E-07    0.448378E-08    0.782089E-08    0.368623E-07    0.118643E-06    0.157492E-06    0.648431E-07   -0.101706E-06   -0.198837E-06   -0.204976E-06   -0.130042E-06   -0.253260E-07    0.101458E-06    0.181088E-06    0.113431E-06    0.127904E-07   -0.742844E-07   -0.964233E-07   -0.702264E-07   -0.230045E-07   -0.109932E-07   -0.257196E-07   -0.792881E-07   -0.121164E-06   -0.719695E-07   -0.282535E-07    0.190828E-07    0.183383E-07    0.424637E-07    0.877042E-07    0.138318E-06    0.926816E-07   -0.657306E-07   -0.201470E-06   -0.206133E-06   -0.485519E-07    0.908666E-07    0.154636E-06    0.138209E-06    0.221218E-07   -0.101907E-06   -0.206998E-06   -0.267027E-06   -0.209599E-06   -0.826628E-07    0.510236E-07    0.134728E-06    0.117194E-06    0.894221E-08   -0.105106E-06   -0.597877E-07    0.100727E-06    0.219950E-06    0.182759E-06    0.233682E-07   -0.144430E-06   -0.217592E-06   -0.169263E-06   -0.517343E-07    0.551317E-07    0.325186E-07    0.243087E-07    0.756872E-07    0.930857E-07    0.159314E-07   -0.128128E-06   -0.211501E-06   -0.169894E-06   -0.467291E-07
   -0.694485E-07   -0.224378E-07    0.221401E-07    0.836066E-07    0.860432E-07    0.491671E-07   -0.669545E-08   -0.696102E-07   -0.129047E-06   -0.937561E-07    0.402438E-07    0.135540E-06    0.149598E-06    0.208053E-07   -0.161068E-06   -0.217021E-06   -0.105352E-06    0.484587E-07    0.120888E-06    0.537277E-07    0.118156E-08    0.269338E-08    0.931195E-07    0.139754E-06    0.513238E-07   -0.639400E-07   -0.106632E-06   -0.440832E-07    0.121775E-07   -0.229683E-07   -0.729073E-07   -0.524572E-07   -0.387171E-07   -0.106852E-06   -0.189536E-06   -0.221171E-06   -0.188663E-06   -0.621178E-07    0.894821E-07    0.193790E-06    0.212407E-06    0.115938E-06   -0.507050E-07   -0.202161E-06   -0.250866E-06   -0.177374E-06   -0.368849E-07    0.999744E-07    0.110361E-06    0.360437E-08   -0.841377E-07   -0.851415E-07    0.689834E-08    0.761140E-07    0.489977E-07   -0.331880E-07   -0.112899E-06   -0.152876E-06   -0.121838E-06   -0.460047E-07    0.559288E-08    0.813540E-08   -0.562482E-07   -0.131840E-06   -0.155442E-06   -0.987869E-07   -0.473705E-08    0.933745E-07    0.157866E-06    0.101932E-06   -0.387892E-07   -0.178805E-06   -0.190619E-06   -0.768422E-07   -0.705869E-09   -0.100708E-07    0.164751E-07    0.376901E-07    0.188341E-08   -0.651132E-07   -0.801101E-07   -0.477235E-07    0.157766E-07    0.685136E-07    0.717596E-07    0.342603E-07   -0.299430E-07   -0.311202E-07   -0.322710E-07   -0.767779E-07   -0.725369E-07    0.154624E-08    0.620740E-07   -0.231309E-07   -0.979338E-07   -0.789127E-07   -0.480766E-07   -0.909703E-07   -0.113284E-06   -0.448274E-07    0.607988E-08   -0.342468E-07   -0.158397E-06   -0.233235E-06   -0.166530E-06   -0.102502E-09    0.143710E-06    0.191821E-06    0.906804E-07   -0.960291E-07   -0.309744E-06   -0.415529E-06   -0.374299E-06   -0.230153E-06   -0.607787E-07    0.325874E-07    0.242335E-08   -0.104976E-06   -0.234904E-06   -0.372155E-06   -0.466619E-06   -0.451784E-06   -0.440951E-06   -0.492843E-06   -0.563039E-06   -0.664022E-06   -0.768607E-06   -0.879071E-06   -0.994832E-06   -0.113137E-05   -0.131484E-05   -0.154962E-05   -0.181444E-05   -0.206876E-05   -0.233224E-05   -0.264476E-05   -0.304648E-05   -0.358880E-05   -0.421438E-05   -0.491228E-05   -0.566852E-05   -0.652931E-05   -0.750234E-05   -0.864049E-05   -0.999422E-05   -0.115371E-04   -0.132961E-04   -0.153198E-04   -0.176468E-04   -0.203251E-04   -0.233983E-04   -0.269381E-04   -0.309954E-04   -0.356498E-04   -0.409827E-04   -0.471101E-04   -0.541450E-04   -0.622301E-04   -0.715006E-04   -0.821184E-04   -0.942863E-04   -0.108208E-03   -0.124133E-03   -0.142378E-03   -0.163257E-03   -0.187111E-03   -0.214352E-03   -0.245448E-03   -0.280933E-03   -0.321407E-03   -0.367525E-03   -0.420052E-03   -0.479862E-03   -0.547904E-03   -0.625257E-03   -0.713146E-03   -0.812933E-03   -0.926145E-03   -0.105450E-02   -0.119991E-02   -0.136452E-02   -0.155070E-02   -0.176111E-02   -0.199866E-02   -0.226663E-02   -0.256862E-02   -0.290861E-02   -0.329099E-02   -0.372060E-02   -0.420273E-02   -0.474317E-02   -0.534827E-02   -0.602492E-02   -0.678059E-02   -0.762339E-02   -0.856206E-02   -0.960599E-02   -0.107652E-01   -0.120505E-01   -0.134733E-01   -0.150455E-01   -0.167799E-01   -0.186896E-01   -0.207886E-01   -0.230909E-01   -0.256113E-01   -0.283645E-01   -0.313655E-01   -0.346293E-01   -0.381705E-01   -0.420036E-01   -0.461421E-01   -0.505989E-01   -0.553857E-01   -0.605128E-01   -0.659887E-01   -0.718200E-01   -0.780110E-01   -0.845631E-01   -0.914752E-01   -0.987426E-01   -0.106357E+00   -0.114307E+00   -0.122577E+00   -0.131146E+00   -0.139990E+00   -0.149080E+00   -0.158387E+00   -0.167874E+00   -0.177481E+00   -0.187163E+00   -0.196930E+00   -0.206713E+00   -0.216406E+00   -0.225977E+00   -0.235359E+00   -0.244496E+00   -0.253333E+00   -0.261810E+00   -0.269868E+00   -0.277451E+00   -0.284502E+00   -0.290960E+00   -0.296767E+00   -0.301891E+00   -0.306333E+00   -0.310082E+00   -0.313062E+00   -0.315028E+00   -0.315914E+00   -0.316116E+00   -0.315914E+00   -0.315028E+00   -0.313062E+00   -0.310082E+00   -0.306333E+00   -0.301891E+00   -0.296767E+00   -0.290960E+00   -0.284502E+00   -0.277451E+00   -0.269868E+00   -0.261810E+00   -0.253333E+00   -0.244496E+00   -0.235359E+00   -0.225977E+00   -0.216406E+00   -0.206713E+00   -0.196930E+00   -0.187163E+00   -0.177481E+00   -0.167874E+00   -0.158387E+00   -0.149080E+00   -0.139990E+00   -0.131146E+00   -0.122577E+00   -0.114307E+00   -0.106357E+00   -0.987426E-01   -0.914752E-01   -0.845631E-01   -0.780110E-01   -0.718200E-01   -0.659887E-01   -0.605128E-01   -0.553857E-01   -0.505989E-01   -0.461421E-01   -0.420036E-01   -0.381705E-01   -0.346293E-01   -0.313655E-01   -0.283645E-01   -0.256113E-01   -0.230909E-01   -0.207886E-01   -0.186896E-01   -0.167799E-01   -0.150455E-01   -0.134733E-01   -0.120505E-01   -0.107652E-01   -0.960599E-02   -0.856206E-02   -0.762339E-02   -0.678059E-02   -0.602492E-02   -0.534827E-02   -0.474317E-02   -0.420273E-02   -0.372060E-02   -0.329099E-02   -0.290861E-02   -0.256862E-02   -0.226663E-02   -0.199866E-02   -0.176111E-02   -0.155070E-02   -0.136452E-02   -0.119991E-02   -0.105450E-02   -0.926144E-03   -0.812932E-03   -0.713146E-03   -0.625257E-03   -0.547904E-03   -0.479863E-03   -0.420054E-03   -0.367527E-03   -0.321411E-03   -0.280938E-03   -0.245452E-03   -0.214352E-03   -0.187105E-03   -0.163248E-03   -0.142373E-03   -0.124133E-03   -0.108200E-03   -0.942762E-04   -0.821162E-04   -0.715186E-04   -0.622636E-04   -0.541650E-04   -0.471019E-04   -0.409410E-04   -0.355945E-04   -0.309679E-04   -0.269117E-04   -0.233668E-04   -0.203025E-04   -0.176514E-04   -0.153232E-04   -0.132668E-04   -0.114652E-04   -0.990197E-05   -0.858666E-05   -0.750755E-05   -0.657503E-05   -0.574872E-05   -0.495997E-05   -0.425290E-05   -0.365134E-05   -0.316995E-05   -0.275030E-05   -0.241386E-05   -0.211714E-05   -0.181348E-05   -0.151020E-05   -0.125212E-05   -0.109924E-05   -0.101567E-05   -0.923766E-06   -0.800592E-06   -0.659055E-06   -0.563865E-06   -0.547127E-06   -0.607095E-06   -0.599770E-06   -0.501492E-06   -0.382448E-06   -0.248864E-06   -0.120767E-06    0.143295E-07    0.521366E-07   -0.853719E-07   -0.276198E-06   -0.390417E-06   -0.365009E-06   -0.172720E-06    0.607951E-07    0.200407E-06    0.205880E-06    0.121420E-06    0.230854E-07   -0.561785E-07   -0.160441E-06   -0.284954E-06   -0.341376E-06   -0.280063E-06   -0.874860E-07    0.860711E-07    0.165057E-06    0.163471E-06    0.724113E-07   -0.385884E-07   -0.649621E-07   -0.744428E-07   -0.140320E-06   -0.201436E-06   -0.121007E-06    0.140203E-07    0.493684E-07    0.896907E-08   -0.465355E-07   -0.396778E-07   -0.294461E-07   -0.620629E-07   -0.849923E-07   -0.135178E-07    0.332284E-07   -0.551766E-08   -0.847366E-07   -0.736400E-07   -0.262724E-08    0.416055E-07   -0.439332E-07   -0.162361E-06   -0.246960E-06   -0.262768E-06   -0.188281E-06   -0.983997E-07   -0.181027E-07    0.590619E-07    0.169370E-06    0.216881E-06    0.120618E-06   -0.292823E-07   -0.150621E-06   -0.218713E-06   -0.196034E-06   -0.112602E-06    0.404000E-07    0.173452E-06    0.120122E-06   -0.161206E-07   -0.111344E-06   -0.172712E-06   -0.178257E-06   -0.797215E-07    0.428559E-08    0.291667E-07    0.121652E-08   -0.450905E-07   -0.490066E-07   -0.640241E-07   -0.453675E-07   -0.103633E-07    0.400694E-07    0.107394E-06    0.141114E-06    0.104180E-06   -0.376312E-07   -0.158066E-06   -0.196057E-06   -0.108702E-06   -0.136346E-08    0.629010E-07    0.888952E-07    0.198508E-07   -0.963591E-07   -0.179076E-06   -0.221706E-06   -0.196567E-06   -0.133490E-06   -0.237550E-07    0.603392E-07    0.873948E-07    0.175689E-07   -0.729129E-07   -0.387151E-07    0.935526E-07    0.171287E-06    0.810731E-07   -0.726993E-07   -0.214405E-06   -0.229693E-06   -0.130410E-06   -0.593380E-08    0.714865E-07    0.233043E-07   -0.173834E-07    0.157161E-07    0.258325E-07   -0.301636E-07   -0.121584E-06   -0.168509E-06   -0.149967E-06   -0.771106E-07

In this next and final singularity example, we get around the problem of needing to port MPI by using the same MPI that's in the container to launch the containers. The trick is this code, which comes at the end of the .bashrc. What it does is effectively replace our shell on the machine with an execution of bash inside the singularity image.

# Put the full path to a singularity image in the file $HOME/sing.txt.
if [ -r $HOME/work/sing.txt ]
then
    IMAGE=$(cat $HOME/work/sing.txt)
fi
if [ "$IMAGE" != "" ]
then
    if [ -r "$IMAGE" ]
    then
        # If the SINGULARITY_CONTAINER variable is set,
        # then we are already in the container
        if [ "$SINGULARITY_CONTAINER" = "" ]
        then
            # Switch to running inside singularity
            exec singularity exec $IMAGE bash --login
        fi
    else
        echo Could not read image file $IMAGE
    fi
fi

In [16]:
!echo /home/jovyan/singu/funwave-tvd.img > ~/work/sing.txt

There's no need to call singularity explicitly, as it's called by each invocation of .bashrc. Note that the funwave-tvd we are executing is the one from inside the image.


In [17]:
!files-upload -F input.txt -S ${AGAVE_STORAGE_SYSTEM_ID} ./
r.runagavecmd(
    "rm -fr output && "+
    "mpirun -np 2 /home/install/FUNWAVE-TVD/src/funwave_vessel && "+
    "tar cvzf singout.tgz output",
    "agave://${AGAVE_STORAGE_SYSTEM_ID}/input.txt"
)


Uploading input.txt...
######################################################################## 100.0%
REMOTE_COMMAND=rm -fr output && mpirun -np 2 /home/install/FUNWAVE-TVD/src/funwave_vessel && tar cvzf singout.tgz output
REQUESTBIN_URL=https://requestbin.agaveapi.co/1131s6o1

 ** QUERY STRING FOR REQUESTBIN **
https://requestbin.agaveapi.co/1131s6o1?inspect

INPUTS={"datafile":"agave://nectar-storage-stevenrbrandt/input.txt"}
JOB_FILE=job-remote-19597.txt
Writing file `job-remote-19597.txt'
OUTPUT=Successfully submitted job 7597476498183155225-242ac11b-0001-007
JOB_ID=7597476498183155225-242ac11b-0001-007
STAT=PENDING
STAT=PENDING
STAT=PENDING
STAT=PROCESSING_INPUTS
STAT=STAGED
STAT=STAGED
STAT=STAGED
STAT=SUBMITTING
STAT=SUBMITTING
STAT=SUBMITTING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=RUNNING
STAT=FINISHED
CMD=jobs-output-get 7597476498183155225-242ac11b-0001-007 fork-command-1.out
All done! Output follows.
Reading file `fork-command-1.out'
======================================================================
 BATHY_CORRECTION DOES NOT EXIST. USE DEFAULT VALUE
 INI_UVZ DOES NOT EXIST. USE DEFAULT VALUE
 WaterLevel DOES NOT EXIST. USE DEFAULT VALUE
 ETA_LIMITER DOES NOT EXIST. USE DEFAULT VALUE
 EqualEnergy DOES NOT EXIST. USE DEFAULT VALUE
 PERIODIC DOES NOT EXIST. USE DEFAULT VALUE
 DIFFUSION_SPONGE DOES NOT EXIST. USE DEFAULT VALUE
 DIRECT_SPONGE DOES NOT EXIST. USE DEFAULT VALUE
 FRICTION_SPONGE DOES NOT EXIST. USE DEFAULT VALUE
 OBSTACLE_FILE DOES NOT EXIST. USE DEFAULT VALUE
 BREAKWATER_FILE DOES NOT EXIST. USE DEFAULT VALUE
 DISPERSION DOES NOT EXIST. USE DEFAULT VALUE
         DISPERSION Default:  DISPERSION
 Gamma1 DOES NOT EXIST. USE DEFAULT VALUE
        Gamma1 Default:  1.0: DISPERSION
 Gamma2 DOES NOT EXIST. USE DEFAULT VALUE
              Gamma2 Default:  1.0: Full nonlinear
 Beta_ref DOES NOT EXIST. USE DEFAULT VALUE
               Beta_ref Default:  -0.531
 Gamma3 DOES NOT EXIST. USE DEFAULT VALUE
                      Gamma3 Default:  1.0: NOT fully linear
 VISCOSITY_BREAKING DOES NOT EXIST. USE DEFAULT VALUE
                   VISCOSITY_BREAKING Default:  SWE Breaking
 SWE_ETA_DEP DOES NOT EXIST. USE DEFAULT VALUE
               SWE_ETA_DEP Default:  0.8
 Friction_Matrix DOES NOT EXIST. USE DEFAULT VALUE
             Friction_Matrix Default:  constant Cd
 Cd DOES NOT EXIST. USE DEFAULT VALUE
                  Cd_fixed Default:  0.0
 Time_Scheme DOES NOT EXIST. USE DEFAULT VALUE
       Time_Scheme Default:  Runge_Kutta
 CONSTRUCTION DOES NOT EXIST. USE DEFAULT VALUE
 HIGH_ORDER DOES NOT EXIST. USE DEFAULT VALUE
  HIGH_ORDER                     NOT DEFINED, USE FOURTH-ORDER
 CFL DOES NOT EXIST. USE DEFAULT VALUE
                       CFL Default:  0.5
 DT_fixed DOES NOT EXIST. USE DEFAULT VALUE
 FroudeCap DOES NOT EXIST. USE DEFAULT VALUE
                 FroudeCap Default:  3.0
 MinDepth DOES NOT EXIST. USE DEFAULT VALUE
                MinDepth Default:  0.1 m
 MinDepthFrc DOES NOT EXIST. USE DEFAULT VALUE
             MinDepthFrc Default:  0.1 m
 SHOW_BREAKING DOES NOT EXIST. USE DEFAULT VALUE
            SHOW_BREAKING Default:  TRUE
 Cbrk1 DOES NOT EXIST. USE DEFAULT VALUE
                    Cbrk1 Default:  0.65
 Cbrk2 DOES NOT EXIST. USE DEFAULT VALUE
                    Cbrk2 Default:  0.35
 WAVEMAKER_Cbrk DOES NOT EXIST. USE DEFAULT VALUE
            WAVEMAKER_Cbrk Default:  1.0
 WAVEMAKER_VIS DOES NOT EXIST. USE DEFAULT VALUE
           WAVEMAKER_VIS Default:  FALSE
 T_INTV_mean DOES NOT EXIST. USE DEFAULT VALUE
             T_INTV_mean Default:  LARGE
 STEADY_TIME DOES NOT EXIST. USE DEFAULT VALUE
             STEADY_TIME Default:  LARGE
 C_smg DOES NOT EXIST. USE DEFAULT VALUE
                     C_smg Default:  0.0
 nu_bkg DOES NOT EXIST. USE DEFAULT VALUE
                    nu_bkg Default:  0.0
 FIELD_IO_TYPE DOES NOT EXIST. USE DEFAULT VALUE
 NumberStations DOES NOT EXIST. USE DEFAULT VALUE
 OUTPUT_RES DOES NOT EXIST. USE DEFAULT VALUE
         OUTPUT_RES NOT FOUND, OUTPUT_RES=1: full resolution
 DEPTH_OUT DOES NOT EXIST. USE DEFAULT VALUE
 Hmax DOES NOT EXIST. USE DEFAULT VALUE
 Hmin DOES NOT EXIST. USE DEFAULT VALUE
 Umax DOES NOT EXIST. USE DEFAULT VALUE
 MFmax DOES NOT EXIST. USE DEFAULT VALUE
 VORmax DOES NOT EXIST. USE DEFAULT VALUE
 MASK DOES NOT EXIST. USE DEFAULT VALUE
 MASK9 DOES NOT EXIST. USE DEFAULT VALUE
 Umean DOES NOT EXIST. USE DEFAULT VALUE
 Vmean DOES NOT EXIST. USE DEFAULT VALUE
 ETAmean DOES NOT EXIST. USE DEFAULT VALUE
 WaveHeight DOES NOT EXIST. USE DEFAULT VALUE
 SXL DOES NOT EXIST. USE DEFAULT VALUE
 SXR DOES NOT EXIST. USE DEFAULT VALUE
 SYL DOES NOT EXIST. USE DEFAULT VALUE
 SYR DOES NOT EXIST. USE DEFAULT VALUE
 SourceX DOES NOT EXIST. USE DEFAULT VALUE
 SourceY DOES NOT EXIST. USE DEFAULT VALUE
 P DOES NOT EXIST. USE DEFAULT VALUE
 Q DOES NOT EXIST. USE DEFAULT VALUE
 Fx DOES NOT EXIST. USE DEFAULT VALUE
 Fy DOES NOT EXIST. USE DEFAULT VALUE
 Gx DOES NOT EXIST. USE DEFAULT VALUE
 Gy DOES NOT EXIST. USE DEFAULT VALUE
 AGE DOES NOT EXIST. USE DEFAULT VALUE
 OUT_NU DOES NOT EXIST. USE DEFAULT VALUE
 TMP DOES NOT EXIST. USE DEFAULT VALUE
      EtaBlowVal Default:  100xmax_depth
 ----------------- STATISTICS ----------------
  TIME        DT
  0.4428E-01  0.4428E-01
  MassVolume  Energy      MaxEta      MinEta      Max U       Max V 
  0.1937E+04  0.1247E+08  0.2996E+01 -0.4596E-06  0.4050E-01  0.4050E-01
  MaxTotalU   PhaseS      Froude 
  0.4050E-01  0.1066E+02  0.3800E-02
   PRINTING FILE NO.     1  TIME/TOTAL:        0.044 /       3.000
 ----------------- STATISTICS ----------------
  TIME        DT
  0.1017E+01  0.4414E-01
  MassVolume  Energy      MaxEta      MinEta      Max U       Max V 
  0.1927E+04  0.1246E+08  0.1900E+01 -0.1727E-05  0.7329E+00  0.7325E+00
  MaxTotalU   PhaseS      Froude 
  0.7338E+00  0.1050E+02  0.6987E-01
   PRINTING FILE NO.     2  TIME/TOTAL:        1.017 /       3.000
 ----------------- STATISTICS ----------------
  TIME        DT
  0.2037E+01  0.4473E-01
  MassVolume  Energy      MaxEta      MinEta      Max U       Max V 
  0.1906E+04  0.1246E+08  0.7393E+00 -0.2109E-05  0.9186E+00  0.9148E+00
  MaxTotalU   PhaseS      Froude 
  0.9215E+00  0.1024E+02  0.8998E-01
   PRINTING FILE NO.     3  TIME/TOTAL:        2.037 /       3.000
 ----------------- STATISTICS ----------------
  TIME        DT
  0.3032E+01  0.4566E-01
  MassVolume  Energy      MaxEta      MinEta      Max U       Max V 
  0.1890E+04  0.1246E+08  0.7188E+00 -0.9413E+00  0.7352E+00  0.6932E+00
  MaxTotalU   PhaseS      Froude 
  0.7378E+00  0.1019E+02  0.7239E-01
   PRINTING FILE NO.     4  TIME/TOTAL:        3.032 /       3.000
 Simulation takes   15.642075522999221      seconds
 Normal Termination!
output/
output/eta_00001
output/v_00003
output/u_00001
output/eta_00003
output/v_00001
output/u_00002
output/eta_00004
output/eta_00002
output/u_00003
output/v_00004
output/v_00002
output/u_00004


In [18]:
!rm -fr output singout.tgz
!jobs-output-get ${JOB_ID} singout.tgz
!tar xzf singout.tgz
!ls output


######################################################################## 100.0%
eta_00001  eta_00003  u_00001  u_00003	v_00001  v_00003
eta_00002  eta_00004  u_00002  u_00004	v_00002  v_00004

In [19]:
# Clean up so that we don't boot into the singularity image without intending to
!rm -f ~/work/sing.txt

In [ ]: